The operation decides the coordinate system

Every essay in this field has ended in the same place by a different route. An area needs a surface. A nearest neighbour needs a metric. A straight segment needs a plane. A degree needs a latitude. In each case the operation was underspecified, a program supplied the missing part from whatever coordinates it had, and the answer was partly a measurement of that choice.

This is the collection, and the question it answers is the one a dataset’s author actually faces: which plane should the file be in?

The three columns #

Every operation in this field reduces to one of three properties of the plane it is performed in.

Areal error decides an area, a density, a proportion. It is the areal scale factor’s departure from one, and it is what makes a shoelace in a conformal plane return 3.06 times the truth over a mid-latitude cell.

Angular deformation decides a shape, a bearing, an overlay, and — surprisingly — a distance ranking, since a plane that shears re-orders candidates at different bearings. It is $\omega$, the invariant, and it is zero exactly when the plane is conformal.

Scale spread decides a distance, a buffer, a threshold. It is how much the scale factor varies over the region, and it bounds how wrong a ranking computed in the plane can be.

The three are not independent — an equal-area plane over a large region must shear, a conformal one must vary in scale — and that dependence is the subject of the whole site. What the table adds is the sizes, over one stated region.

The scores #

Measured over 10° west to 30° east and 35° to 60° north, at 49 sample points, from each projection’s own derivatives:

Three things in that table are worth reading slowly.

The conformal conic wins two columns. Its angular deformation is at the noise floor, which is what conformal means, and its scale spread is 6.4 per cent — an order of magnitude better than anything else here, because a conic fitted to a mid-latitude band is the classic answer for exactly this region. It is 11.7 per cent out on area.

The equal-area members are exact on area — three and five parts in $10^{12}$, which is the arithmetic and not the projection — and are 19° and 39° out on shape. There is no partial credit in that column: a projection is equal-area or it is not.

And Web Mercator is not the best at anything. Its angular deformation is 0.2585°, not zero, because it applies spherical formulae to geodetic latitudes — the site’s headline, Web Mercator is not conformal, arriving here as a fifth-place finish in the column it is named for. The genuinely conformal candidate beats it by five orders of magnitude in the same measurement.

The theorem in the table #

The assertion the site’s gate carries about this table is not that these five have no common winner. It is that no set of candidates ever will.

A projection exact in two of the three columns has $ab = 1$ from the areal column and $a = b$ from the angular one, which forces $a = b = 1$: the scale factor is one in every direction at every point, so the map is an isometry from the sphere to the plane. No map is faithful forbids it, by computing the Gaussian curvature two independent ways and finding that it is intrinsic — Gauss’s Theorema Egregium, exercised on every build of this site rather than cited.

So the table’s structure is guaranteed in advance. Any list of candidates, over any region, has its winners split across the columns, and the assertion is written to fail if one ever did not — which would mean either an arithmetic error or a projection that beats a theorem.

What each operation actually costs #

The columns are properties of planes. Here are the operations, with what using the wrong plane costs, every figure computed by the machinery of the essays above rather than estimated:

Four operations, four different exact answers, and not one of them is the plane a dataset is likely to be stored in.

Two operations that want opposite planes #

The table’s abstraction becomes concrete when two operations on the same dataset pull in different directions, which is the ordinary case rather than a corner one.

A dataset of administrative regions is shaded by density, which needs an area, which needs an equal-area plane. The same dataset is drawn on a screen and panned, which needs conformality — as the pyramid did not have to be Mercator measures, the shear cost of the alternative is 13.8° at 60° north, and every symbol and every piece of imagery inherits it.

Those are the two columns at opposite ends of the table, and a single stored plane can serve one of them. What resolves it is not a compromise projection — that gives 19° of shear and an areal error, losing both — but a separation of concerns: compute the areas once on the surface, store them as attributes, and choose the stored plane for the drawing.

The general form of that resolution is the field’s most portable conclusion. Where two operations want different planes, the one that produces a number should be moved out of the pipeline and computed exactly, once; the one that produces a picture keeps the plane. Numbers travel as attributes and pictures do not.

The table’s best all-round candidate holds up as an instrument as well as a score. A bar drawn at 45° on it reads within a few per cent across the whole band, which is what a 6.4 per cent scale spread means in the units a reader would notice, and why a conic wins the distance column.

The one column with no exact answer #

Area has an exact plane. Shape has an exact plane. Distance has none, and the asymmetry is worth stating because it is not obvious from the table’s layout.

An equidistant projection is true along particular lines — from one point in the azimuthal case, along the meridians in the conic case — and there is no projection with a scale factor of one everywhere, because that is the isometry again. So the scale-spread column has a best entry and never a zero, and every distance is either computed on the surface or approximated with a bound.

That is why the practice field integrates a line’s scale factor rather than sampling it, finding 18 millimetres on one line and 97 metres on a shorter one; why a nearest-neighbour query needs a filter width rather than a projection; and why the buffer question has no clean answer at all. Two of the three properties can be had exactly and the third cannot, which is a sharper statement of the trade-off than the usual conformal-or-equal-area framing.

The stored plane is not the only decision #

One more distinction the table cannot show: a dataset has a stored plane, and it may also have a working plane and a display plane, and the three need not agree.

The vector case makes this cheap. Reprojecting coordinates is exact, so a pipeline can store in one plane, compute in another and draw in a third, paying only arithmetic. The raster case does not: every warp resamples, at first, second or third order depending on the kernel, and reprojecting a raster invents values measures what the round trip costs.

So the advice divides by data type, which is the last thing this field has to say and is not something a table of projections could contain. For vector data, choose the plane per operation; for raster data, choose it once and keep the original.

The rule this field arrives at #

The table does not say choose the best projection, because there is no such thing and which projection is best is the essay on why the question is malformed. It says something more useful and more specific.

Store in whichever plane makes the dominant operation exact. If the dataset exists to be shaded by density, store it equal-area and the areas are free. If it exists to be drawn and panned, store it conformal and the shapes are free.

Compute the rest with a bound rather than in the stored plane. Every operation in this field has one: a nearest-neighbour query filtered in the plane and decided on the ellipsoid is exact if the filter’s width exceeds the scale spread; a length is a scale factor integrated along the route; an area is a closed form.

And never measure off the picture. The most consistent finding of this field is that the display projection is the worst possible place to compute anything, because it was chosen for a reader’s eye. Compute on the surface, store the answer as an attribute, and let the picture be a picture.

One qualification the table needs is that it is a table about a region. Scored over two regions rather than one, the same candidates do not keep their order: the ranking is not a property of the projections, and a table like this is a measurement of the ground as much as of the plane.

What the field measured, collected #

Fifteen essays, two ladders, and one recurring shape: an operation is performed in a coordinate system, the system supplies a metric nobody chose, and the answer carries it. The sizes, in one place:

• a resolution table read as ground distance: wrong by sec φ, a factor of 2 at 60°;
• a printed scale, at the same latitude: a 1:136,495 map is at 1:68,765;
• a scale bar carried up its own sheet: 74 per cent short at 75°;
• the poles a square world discards: 1,901,487 km², where a two-tile root would have dropped 3,558;
• a vertex rounded to a pixel: up to 16.8 m at zoom 12, and a merge rate of 12 per cent decided by the grid’s phase;
• a feature clipped to nine tiles: nine labels, up to 1,863 km from the right one;
• one pair of numbers on four datums: 103, 138 and 195 metres apart, and 7,942 kilometres in the other order;
• a degree buffer at 60°: an ellipse of 1.9966:1, which is not quite sec φ and is not meant to be;
• an area computed in degrees: 1.75×; in a conformal plane: 3.06×; on a sphere rather than the ellipsoid: 0.45 per cent;
• a straight segment across the Atlantic: 718 km from its route, with a fitted exponent of 2.001;
• a shape across the antimeridian: a bounding box 18 times too wide and a midpoint at the antipode;
• a nearest-neighbour query over Europe: 5 of 400 wrong, by up to 24 km;
• a raster warped and returned: first, second and third order, and no correct answer at all.

Every one of those exists with perfect input data. Nothing here is an observation, an instrument or an error model — that is the practice field’s subject, and the test that separates the two is that every measurement above survives being handed exact coordinates. What is being measured is the operation.

All of it sits inside the site’s founding measurement. Every projection in the library is scored for angular deformation and areal error and plotted, and the empty corner near the origin is the claim that matters — the same claim this essay’s table makes with five points instead of twenty-one.

The same argument, one field over #

The practice field reached this conclusion from the other side and it is worth putting the two sentences beside each other.

There, a coordinate is the end of a chain: an instrument reading, four corrections that do not commute, two conventions that are not measurements, and a number written on a drawing. What a tape measures prices the chain; the tolerance decides the model says which corrections a stated tolerance permits dropping; what a grid is made of lists the five declarations the number does not carry.

Here the number arrives perfect and is used, and the same five declarations turn out to decide the answer to every question asked of it. The two fields are the two halves of one statement: a coordinate is meaningless without its conventions, whether it is being produced or consumed, and both halves have the same list of conventions in them.

What differs is where the tolerance comes from. A survey’s tolerance is stated in a specification and the corrections are compared against it. An operation’s tolerance is usually never stated at all, which is why this field’s essays measure sizes rather than argue about acceptability — the size is available, and whether it matters is a question only the person asking can answer.

The practice field’s version of this essay’s table inverts each correction to the line length at which it alone reaches a stated tolerance. The shape of the argument is identical — an operation, a plane and a size — and the difference is that a survey states its tolerance in advance and a query almost never does.

What is left for later #

Three things this field named and did not take.

Spatial indexing as a cost. The structures — the quadtree, the R-tree, the discrete global grid — are algorithms-data-structures.com’s subject. What was owned here is the geometry a tiling must satisfy, and the containment property of a quadkey, which is a statement about rectangles rather than about search.

Generalisation. Simplification and smoothing are a subject with their own literature. Densification, their inverse, is a projection question and is here; the forward direction is not.

And the deflection of the vertical, still a citation under this collection’s standing ruling on the geoid — which means the vertical half of every coordinate in this field is stated parametrically rather than computed. That deferral is an old one and is recorded again rather than quietly dropped.

What naming the plane would look like in practice #

The rule is easy to state and the interesting question is what obeying it costs, because the answer decides whether it is advice or a proposal.

Nobody currently chooses the plane. The library does. An operation is performed on whatever coordinates the file holds, in whatever plane they were stored in, by code whose default is to say nothing about it. So the plane is decided by the last person to convert the data, for reasons that had nothing to do with the operation being performed — which is the mechanism behind every failure this field measures.

Obeying the rule means transforming per operation rather than per dataset. Store the geometry in the plane it was captured in, and for each operation move it to the plane that makes that operation exact: an ellipsoidal computation for an area, a conformal local plane for a bearing, the ellipsoid itself for a distance. That is one transformation per operation instead of none, which is a real cost on a large dataset and a negligible one on a small one.

And the table above says when it can be skipped. Every column carries the cost of using the wrong plane, so a user with a stated tolerance can read off whether their stored plane is good enough for their operation. Most of the time it is, which is why the practice survives — the rule is not that everybody must transform, but that everybody must be able to say why they did not.

The recordable half is cheaper still. A derived layer that carried the plane it was computed in — one string, in the metadata, beside the coordinate system it is stored in — would let a downstream consumer answer the question at all. At present there is nowhere to put it, which is why a slope raster, a curvature raster, a hillshade and a compactness score all arrive with no indication of the frame their numbers live in.

Two operations wanting opposite planes is the case that makes the metadata unavoidable, since no stored choice serves both and the file will therefore hold layers computed in different planes. Without a record, the layers are indistinguishable; with one, the conflict is visible and can be handled.

None of that requires a new format, only a convention about what goes in the description field every format already has.

Where the whole thing points #

The site’s founding rule is that no projection is called conformal until the angular deformation has been measured and found to be zero. This field’s rule is the same sentence with the subject changed: no operation is called correct until the plane it was performed in has been named and the cost of that plane has been measured.

Both are the same instinct — that a property is a measurement rather than a label — applied once to a map and once to what a machine does with one. The map’s version has been the site’s business since foundation. The machine’s version is newer, is where almost all cartography now happens, and turns out to have the same answers, because the geometry did not change when the medium did.