Concept

Quantisation — where it appears

Rounding a position onto a grid of representable places — a file's decimal places, or a screen's pixels — and the displacement that costs. It happens at least three times in a mapping pipeline — in the written coordinate, in the projected grid, and in a vector tile's integer lattice.

Named by 12 essays across 3 fields — each of them below, with the objects they name alongside it.

One pixel at zoom 11: 76.4 m projected, 47.6 m at 51.5°. The grid squares are pixels, at their own size. The open mark is the stored coordinate and the filled one is where it is drawn: rounding moves it 20.2 metres, against a worst case of 33.6 — half a pixel's diagonal, which is the whole of the bound. The second point sits 43 metres away, 0.58 of a pixel east and 0.70 north, so whether the two are drawn as one dot or two is decided by where the tile grid happens to fall: they merge for 12 per cent of the possible offsets, against the 12 per cent the two fractions predict.

The pixel is a place with a size

Drawing a coordinate rounds it to a pixel, which moves it by up to half a diagonal — 16.8 metres at zoom 12. Whether two points 43 metres apart appear as two dots is not a property of the data at all: they merge for 12 per cent of the positions the tile grid could take, and the closed form predicts 12.4.

applied · Screen
A feature across nine tiles at zoom 4, and the nine labels it gets. A rectangle from -10° to 30° east and 36° to 62° north, drawn in the projected plane with the tile boundaries over it. A renderer that draws each tile from the geometry inside that tile puts the label at the centroid of the piece, marked hollow; the whole feature's centroid is the filled mark. The furthest piece's label is 1863 kilometres from it. The pieces' areas sum to the whole to 2.2e-16 of a relative part, so the clipping is exact and the displacement is the operation rather than an error in it.

A tile is drawn without its neighbours

Independence is what makes the scheme scale: one request touches one square of the world. It also means a feature crossing nine tiles is nine features, each labelled at its own centroid — up to 1,863 kilometres from the whole one — and the displacement is bounded by the size of the feature rather than the size of a tile, so it grows as the reader zooms out.

applied · Screen
A raster warped to Lambert azimuthal equal-area and back, nearest against bilinear. The left panel is the field the raster carries — a smooth analytic function, so that the error of an interpolation is the interpolation's error and not a photograph's history. The other panels are what is left after warping into Lambert azimuthal equal-area and back to Equirectangular, shown as the difference from the original at six times the contrast. Nothing moved: the coordinates go through the maps exactly. What is lost is that a target pixel's centre does not fall on a source pixel's centre, so a value has to be invented for it. Bilinear is closer to the field — RMS 0.0022 against 0.0212 — and has given up 0.63 per cent of its variance to get there. The panels are drawn at 48 by 32 cells; the measurement is made at the same resolution.

Reprojecting a raster invents values

Moving a picture from one projection to another moves no coordinate — the maps are exact both ways. What is lost is that a target cell's centre does not land on a source cell's centre, so a value has to be made up for it, and the making-up has an order of convergence: 1.00 for nearest, 1.98 for bilinear, 2.93 for a cubic, measured by refining the grid.

applied · Dataset
The scale asked for and the scale the pyramid has, at 0°. A tiling scheme exists only at integer zoom levels, a factor of two apart in resolution, so a request for any scale between them is answered by the nearest rung. The ratio runs from 0.707 to 1.405 — 1/√2 to √2 — and repeats identically at every doubling, which is four times that in area. A request for 1:10,000 is served at zoom 16, which is 1:8,531; A request for 1:25,000 is served at zoom 14, which is 1:34,124; A request for 1:50,000 is served at zoom 13, which is 1:68,247. Nothing anywhere reports it, because the map that arrives is a perfectly good map of something.

Zoom is a ladder

A tiling scheme exists only at integer zoom levels a factor of two apart, so a request for 1:25,000 is answered with 1:34,124 — 36 per cent coarser, and 86 per cent coarser in area. The mismatch runs from 1/√2 to √2 and repeats identically at every doubling, and nothing anywhere reports it, because the map that arrives is a perfectly good map of something.

applied · Screen
What the 5th decimal place of a coordinate is worth on the ground. A latitude written to 5 decimal places steps 1.112 m north for one unit in its last digit, at every latitude, because the meridian does not care where it is measured. A longitude written to the same 5 places steps 1.112 m east on the equator and 0.097 m at 85°, because a degree of longitude is a degree of a circle whose radius is R cos φ. The same written precision means two different distances at the same point, and a different pair at every other.

A coordinate is a number with a width

Every number on this site so far has been exact. A written coordinate is not: five decimal places of a degree is 1.112 metres of latitude everywhere and 1.112 metres of longitude only on the equator, falling to 0.097 at 85°. The same written precision carves the ground into a cell that is square in one place and eleven times longer than it is wide in another.

distortion · Precision
What a 3-figure grid reference names. The square is the 100-metre cell a 3-figure reference stands for, and the filled dot is the point the reference was made from. A reference is TRUNCATED rather than rounded, so the digits name the square's south-west corner — the open dot — which here is 9.1 metres west and 82.8 metres south of the point. Rounding would have landed on the nearest corner instead, 9.1 metres away east–west. The truncation is deliberate: it is what makes a shorter reference a larger square containing the same point.

A grid reference names a square

This ladder has priced everything about a grid except how a coordinate on it is written. A grid reference is truncated rather than rounded, so it names the south-west corner of a square rather than a point in it — and a population of references is displaced half a cell each way, which is a bias rather than scatter and does not average out.

practice · Grid
The same kernels, across an edge. Each kernel measured over the same five rasters as the smooth measurement, on a field that carries a step across a tilted line instead of a smooth function. The solid lines are the edge and the faint ones the smooth field. The orders on the smooth field are nearest 1.00, bilinear 1.98, cubic 2.93; across the edge they are nearest 0.78, bilinear 0.58, cubic 0.60 — within a factor of 1.33 of one another, and the fastest of them belongs to nearest-neighbour, which does no interpolating at all. The ranking a smooth field establishes does not survive a discontinuity, and a real raster is mostly edges.

An edge has no order of convergence

On a smooth field the three resampling kernels converge at orders 1, 2 and 3 and the choice is obvious. Across a discontinuity they converge at 0.78, 0.58 and 0.60 — within a factor of 1.4 of each other, in an order that puts nearest-neighbour first, and a real raster is mostly edges.

applied · Dataset
Equal bands of screen, unequal bands of ground. A section through a camera pitched 60° from straight down, at the height that makes an untilted view exactly the flat map — 1152 pixels above the plane for a 768-pixel screen and a 36.87° field of view. The rays are drawn at equal spacings up the screen and the blocks below them are the ground each of those equal bands covers: 1.75 plane units per pixel at the bottom of the frame and 16.88 at the top, a factor of 9.7. Nothing about this is a projection of the sphere; it is a photograph of a map, and the map underneath it is still Web Mercator.

A tilted view has no zoom level

Every essay about the screen so far assumes the map lies flat on it, which was true until about 2015. Pitch the camera sixty degrees and one frame asks for 3.6 zoom levels at once, a square tile covers ground four and a half times deeper than it is wide, and the pyramid has one integer per tile to answer with.

applied · Screen
One unit of a vector tile, in metres of ground. A vector tile's coordinates are integers on a lattice 4096 units across the tile, and the tile halves at every level, so one unit is a distance that halves too: 5.48 m at z10 and 0.086 m at z16, at 55°. It is also a different distance at every latitude, by cos φ, because the tile is in Web Mercator — the same factor that makes a grid metre a different quantity of ground at every latitude, arriving in the file format rather than in the projection.

A vector tile has an integer grid

Six essays on this ladder treat a vector tile as the thing a raster tile is not: geometry, resolution-free, styled at draw time. Its coordinates are integers on a lattice 4,096 units across a tile, the tile halves at every level, and at 55° north one unit is 88 metres at zoom 6 and 21 millimetres at zoom 18.

applied · Screen
One of these three lines does not slope. Three lengths at each zoom, for a world coordinate near the antimeridian. The pixel halves with every level, as it must. The spacing between representable double-precision values is far below it and halves with it. The spacing between representable SINGLE-precision values does not move at all — 2.0 metres at every zoom, because it is a property of the size of the number and the world does not get smaller. The two cross at zoom 17, and past it a vertex snaps to a lattice coarser than the pixels it is drawn into.

The renderer runs out of numbers before the zoom does

Eleven essays price the pyramid in exact arithmetic. The pipeline that draws it carries single precision, where a world coordinate near the antimeridian quantises to two metres — at every zoom, because the number does not get smaller when the pixel does. At zoom 22 that is fifty-four pixels.

applied · Screen
How far the two routes end up apart. The distance between a line simplified directly at the final tolerance and the same line simplified through two intermediate products, as a multiple of the final tolerance, with a stated extra step applied to each intermediate. With nothing in between the two are the same line to the last bit, because Douglas–Peucker's outputs are nested. Rounding the intermediate to half the tolerance, smoothing it for legibility, or running it through a moving average each break that, and the last of them puts the final product 0.142 away from where a direct route would have put it — nine times the tolerance the product is published under.

Two routes to one scale

A national series is cascaded — the million is derived from the quarter-million, which was derived from the fifty — and the folklore is that the errors accumulate. They do not: Douglas–Peucker and Visvalingam both cascade to the same line the direct route produces, bit for bit, because both output a sublevel set of a per-vertex number. What breaks it is anything else in the chain, and a moving average puts the product nine tolerances away.

applied · Generalise
Two places, one written coordinate. The lattice a coordinate written to 5 decimal places lives on, at 45° of latitude, where a cell is 1.11 metres north to south. Two real places a fifth of a cell apart are drawn as open marks and the single value they both round to as a filled one. The distance between them, computed from what was written, is exactly zero — an error a noise model cannot produce, since independent noise never puts two different points in the same place.

Rounding is not noise

Eight rungs treat a coordinate's error as noise that averages down. A published coordinate has a second error that does not: it is deterministic, it is shared between every point in the same cell, and at five centimetres apart ninety-three per cent of pairs come out as the same place — which no amount of independent noise can produce.

distortion · Precision

Named alongside it

The objects these essays reach for when they reach for this one.

ToleranceConventionGround resolutionPrecisionVerificationTile pyramidWeb MercatorZoom levelScale factorClosed formConvergence orderFalse origin

All concepts