Dataset — the ladder · Cartographic Projection

53° north, 2° west: four places one pair of numbers could be. The same two numbers, read as three datums and as the pair in the other order. The centre is the WGS84 reading; the marks are where the ground is if the numbers were measured against OSGB36, ED50, NAD27 instead, at 103 m, 138 m, 195 m. Every one of those is a plausible position — near enough to look like a survey difference and far enough to be a different field. Reversing the two numbers instead moves the point 7942 kilometres, and is the mistake nobody worries about because it is usually obvious.

A coordinate without its system is not a location

The same two numbers name ground 103 metres apart on one datum, 195 on another, and 7,942 kilometres away if the pair is read in the other order. The datum errors are the dangerous ones because they are plausible — and the axis swap, which everyone calls obvious, does nothing at all along a line that crosses three continents.

rung 1 · applied

"Within 0.01°" on the ground, at five latitudes. The same condition drawn at five latitudes, all at one scale. A degree of latitude is a fixed distance — 1106 metres here, varying by less than one per cent from equator to pole — while a degree of longitude collapses as cos φ, from 1113 metres to 289 at 75°. So the "circle" is an ellipse of 3.86:1 there, and it encloses 26 per cent of the ground the same condition covers on the equator. Even on the equator it is not round: M is smaller than N by the flattening, so the shape is 6694 parts per million shorter north–south than east–west.

A degree is not a unit of length

"Within 0.01 degrees" is a condition anybody can write and no instrument can measure. On the ground it is an ellipse — 1,106 metres north–south and 558 east–west at 60° — and even on the equator it is not a circle, because the meridian's radius of curvature is smaller than the parallel's by the flattening.

rung 2 · applied

The area of one 20° × 10° cell at 50–60° north, seven ways. The cell has an exact area — R²Δλ(sin φ₂ − sin φ₁), 1,416,580 square kilometres — so every other row is a measurement of the method rather than of the ground. The equal-area projection returns it to 1.000000 and the spherical polygon formula to 1.000000, which is three routes agreeing — and the same cell integrated on the ELLIPSOID comes out 0.45 per cent away from all three, because the sphere is a model. Taking the shoelace in Mercator gives 3.06 times too much, and treating degrees as a length gives 1.75 times — about sec φ at the cell's middle, which is where that error comes from.

Computing an area needs a surface

A shoelace over a ring of coordinates returns a number whatever the coordinates are. For one twenty-by-ten-degree cell it returns 1.75 times the true area in degrees, 3.06 in a conformal plane, and exactly the closed form in an equal-area one — and the closed form itself is 0.45 per cent out, because the sphere is a model too.

rung 2 · applied

A straight segment is a claim about a plane

Two exact endpoints, joined by a straight line in the plane the file is stored in. On the ground the line is 718 kilometres from the route it claims between New York and London, and 2,961 between London and Tokyo. The departure grows as the square of the length — fitted exponent 2.001 — so a stated tolerance costs vertices as a square root.

rung 3 · applied

A 20° shape across the antimeridian, in the space where the numbers live. Longitude runs across the page from −180° to 180°, which is where the failure is: the shape is one rectangle on the ground and two pieces in the numbering, and every operation that treats longitude as a real number sees the two. The bounding box comes out 359° wide instead of 20°, the planar area comes out 17 times too large because the shoelace encloses the complement, and the midpoint of a segment from one edge to the other lands 20015 kilometres away — the antipode of where it belongs. The true area, from the closed form, is 4,920,667 square kilometres.

The antimeridian is a cut in the numbers

A twenty-degree box across 180° has a bounding box of 359.4°, a planar area seventeen times too large, and a midpoint 20,015 kilometres from where it belongs — which is the antipode, exactly. Moving the cut moves the failure and never removes it, because a circle cannot be numbered by an interval.

rung 3 · applied

Where the middle of a 20° × 20° region is, in five planes. The region is drawn in longitude and latitude — which is itself a projection, and one of the ones being compared. Each filled mark is the shoelace centroid computed in one projected plane and inverted back to the ground; the hollow mark is the centre of area on the sphere, by integration. They spread over 273 kilometres. The equal-area member is 66 kilometres out, because a centroid is a first moment and preserving area says nothing about where the area sits.

A centroid belongs to a plane

Every renderer labels a region at its centroid, and every centroid is a shoelace over coordinates as stored — which is a statement about the plane they are in. Six planes put the middle of one 20° × 20° region up to 273 kilometres apart, the equal-area member is 66 kilometres out, and the disagreement falls as the square of the region's size.

rung 3 · applied

5 of 400 nearest-neighbour queries change answer in the plane. 40 sites and 400 queries over -20° to 40° east and 35° to 70° north, each query answered twice — once by geodesic distance and once by straight-line distance in the stored plane. They agree 98.8 per cent of the time, which is why the operation survives, and the 5 that differ are marked. The mechanism is not that the plane is wrong by a lot but that its scale factor varies: over this region it spans 139 per cent, and every disagreement is a contest closer than that — the worst margin measured is 6.4 per cent. The furthest a wrong answer is from the right one is 24 kilometres.

Nearest is a question about the metric

Asked in the plane the data is stored in, a nearest-neighbour query returns a different site for 5 of 400 queries over Europe on Web Mercator — and 103 of 400 on the plate carrée, with the wrong answer up to 309 kilometres further away. Every disagreement is a contest closer than the region's own scale spread, and over a city there are none.

rung 4 · applied

Reprojecting a raster invents values

Moving a picture from one projection to another moves no coordinate — the maps are exact both ways. What is lost is that a target cell's centre does not land on a source cell's centre, so a value has to be made up for it, and the making-up has an order of convergence: 1.00 for nearest, 1.98 for bilinear, 2.93 for a cubic, measured by refining the grid.

rung 4 · applied

Two readings of the same triangle, and the band between them. The same three vertices, joined two ways: with straight lines in the plane the coordinates are stored in, and along the ground. Every dot is a point the two readings disagree about — inside on one and outside on the other. The band covers 29.5 per cent of the polygon, which is 4241 thousand square kilometres, and it is not an error in either reading: the file does not say which one it means. Densifying the stored boundary removes it, which is the only fix there is. Drawn in Equirectangular.

Inside is a claim about the edges

Whether a point is inside a polygon is not a property of the point and the polygon. It is a property of the plane the edges were understood to be straight in, and between two readings of the same file there is a band of disagreement — 29 per cent of one triangle's area, 4.2 million square kilometres, and the file does not say which reading it means.

rung 4 · applied

The operation decides the coordinate system

Five candidate planes over one region, scored on the three things a spatial operation depends on. The conformal conic wins shape and distance and is 11.7 per cent out on area; the equal-area member is exact on area and 38.9° out on shape. No candidate is exact in two columns, and no candidate ever will be, because one that was would be an isometry.

rung 5 · applied

The same kernels, across an edge. Each kernel measured over the same five rasters as the smooth measurement, on a field that carries a step across a tilted line instead of a smooth function. The solid lines are the edge and the faint ones the smooth field. The orders on the smooth field are nearest 1.00, bilinear 1.98, cubic 2.93; across the edge they are nearest 0.78, bilinear 0.58, cubic 0.60 — within a factor of 1.33 of one another, and the fastest of them belongs to nearest-neighbour, which does no interpolating at all. The ranking a smooth field establishes does not survive a discontinuity, and a real raster is mostly edges.

An edge has no order of convergence

On a smooth field the three resampling kernels converge at orders 1, 2 and 3 and the choice is obvious. Across a discontinuity they converge at 0.78, 0.58 and 0.60 — within a factor of 1.4 of each other, in an order that puts nearest-neighbour first, and a real raster is mostly edges.

rung 5 · applied

One edge is not an edge

The three resampling kernels were measured across a discontinuity and came out at 0.78, 0.58 and 0.60 — one straight edge at 27° to the graticule. Across thirteen edges the same kernels span 0.19 to 0.87, the ranking between them reverses, and for an edge lying along a parallel the error does not fall with refinement at all.

rung 6 · applied

Give the edge a width and the kernels get their orders back. Every edge this collection has resampled across has been exactly discontinuous, which is not what a sensor produces: a footprint, an atmosphere and a lens all smooth a boundary over a cell or two before anything is resampled. Convolving the edge with a Gaussian of stated width and refitting gives 1.23, 1.97 and 3.60 at one degree of blur, against 0.78, 0.58 and 0.60 with no blur at all. The blur is held fixed in degrees while the grid refines, which is what happens to a real sensor's data as its resolution improves.

A real edge has a width

Thirteen edges were measured and every one of them was exactly discontinuous, which no sensor has ever produced. Convolving them with a point-spread function of one degree — a cell or two — takes the three kernels from 0.78, 0.58 and 0.60 back to 1.23, 1.97 and 3.60, and takes the edge along a parallel, which converged at −1.49, up to 1.92 for bilinear and 3.73 for cubic.

rung 7 · applied

A ring round the pole at 80°, and the two pieces it makes. the boundary of a small polar cap — and of everything else. A closed curve divides a sphere into two pieces and neither of them is the outside: one is 4 thousand square kilometres and the other is 506 thousand, a ratio of 130.6 to one, and the coordinates are the same either way. The two colours are the two pieces, sampled at points rather than shaded, because shading one of them would already be the decision this figure is about.

A polygon on a sphere has no outside

Seven essays have treated a stored ring as a boundary between inside and outside. A closed curve on a sphere divides it into two pieces and neither of them is the outside, so every polygon in every file depends on a convention that no coordinate carries — and the two conventions in common use disagree by a factor of fourteen on any ring that contains a pole.

rung 8 · applied

What simplifying a boundary does to the number stored beside it. One region, simplified at five tolerances, with the error in the two quantities a consumer computes from the pair. If the density was stored, the total it implies moves by exactly the area's error — -1.55 per cent at the loosest tolerance. If the total was stored, the density it implies moves the other way by the same amount. Nothing in the file says which of the two was measured and which is being derived, and the simplification is normally done by a tool that never opens the attribute table.

The attribute is a claim about the geometry

Fourteen essays price what a stored coordinate means and not one asks what the number stored beside it means. A rate is a quantity divided by an area, the area belongs to the geometry, and no format records which area — so a simplification that moves the outline by nothing visible moves the implied total by 1.55 per cent, an unweighted average of densities is 4.09 per cent out, and a choropleth gives a polar square kilometre fifteen times the ink of an equatorial one.

rung 9 · applied

The query, the box it becomes, and the 14 per cent it loses. A query for everything within 400 kilometres of a place at 55° north, drawn in the stored coordinates of Web Mercator. The ring is the true answer's edge; the rectangle is the box the index is given, sized from the scale factor at the query point. The arcs outside the rectangle are true neighbours the index never returns — 13.6 per cent of the rim, reaching 5.0 per cent of the box's own half-width beyond it — and nothing downstream can tell they are missing.

The query a fast path actually answers

Fifteen rungs price what a stored coordinate means and none asks what it is searched with. No index answers "within two hundred kilometres of here"; an index answers "inside this rectangle of stored coordinates", and the rectangle is built by somebody's arithmetic — which at 55° north silently drops a tenth of the true answer on a conformal projection and three fifths of it on an equal-area one.

rung 10 · applied

One rule, applied once, to one dataset. Two features 3 metres apart on the ground, and a snapping tolerance of 5 map units on Web Mercator. The tolerance's reach is the curve; the pair's separation is the line. They merge everywhere below about 52.8° and stay separate above it — the same features, the same rule, the same run. A pipeline that cleans a global dataset in one pass produces a topology that changes at a latitude nobody chose and nothing records.

A tolerance in map units is not a tolerance

A snapping tolerance is a number, and the number is in whatever units the file is in. Five map units on Web Mercator is 4.97 metres of ground at the equator and 0.87 at eighty degrees — so a rule that merges two features three metres apart merges them everywhere below 52.8° north and refuses everywhere above it, in one pass, over one dataset, with nothing recording where the boundary is.

rung 11 · applied