When the answer is not in the library

The identification method is a ranking: fit every candidate to a map’s graticule, remove the plane similarity each one needs, and sort by what is left. On the maps it has been given it works spectacularly — the right candidate wins by a factor of 3.5 × 10⁷, and a conic’s cone constant comes back as 0.70711 without being told the latitude.

The taught rule for choosing a projection was scored by asking how often it was right. This method has never been scored at all, because it has never been given a question whose answer is nothing.

A ranking always produces a winner. That is what makes the number above worth nothing on its own: a method that cannot say none of these is a ceremony, and the way to find out which it is, is to take the answer away.

What the impostors are #

Doing this for every candidate in turn, each held out of its own library, produces twelve wrong answers, and none of them is random:

Every impostor is the nearest thing in the library, which is the separation measurement read as a list of neighbours. Mercator is drawn as a conformal conic with the cone constant pushed towards one — the limiting case in which a cone becomes a cylinder. Eckert IV is named as Mollweide, which is the other elliptical pseudocylindrical. Miller and the equirectangular name each other, which they should, since Miller is the equirectangular with a stretched latitude.

That is the method behaving well, not badly. It is finding the closest available description, which is exactly what it is for. The problem is only that it reports the description as an identification — the same conflation Web Mercator is not conformal is about, where a name that describes a construction is read as a statement about a property.

The residual will not do the work #

The obvious rejection rule is a threshold on the residual: name the winner only if it fits to better than some tolerance. It does not work, for a reason worth being precise about.

A residual is a length — here a fraction of the map’s own width — and whether a given value is small depends entirely on how accurately the map was measured. A residual of 0.4 per cent is enormous for a map measured to a micrometre and is beneath the noise for one measured off a photocopy. A threshold on the residual is a threshold on a quantity whose scale is set by something outside the measurement.

The margin has no such problem. It is the ratio of the runner-up’s residual to the winner’s, so any common factor — the map’s size, the measurement precision, the units — cancels.

The two populations do not merely separate; they separate by a factor of two million, with no overlap. Any threshold between 2 and a million names the projection every time it is present and refuses every time it is not, on all twelve candidates.

Why the gap is that large #

The mechanism is worth stating because it says when the rule will fail.

With the truth present, the winner’s residual is the measurement noise, because the model is exactly right and there is nothing else left over. With exact synthetic data that is the arithmetic’s floor — 10⁻¹⁵ of the map’s width — so the margin is the runner-up’s residual divided by nothing at all.

With the truth absent, the winner’s residual is the shape difference between the map and the nearest available projection, and the runner-up’s is the shape difference to the next nearest. Those are two similar numbers, so the ratio is close to 1.

The margin is therefore not measuring goodness of fit. It is measuring whether the residual is dominated by noise or by model error, which is exactly the question is the model right — and it is the same statistic an analysis of variance uses, arriving in cartography by a different road. It is also the reason a tolerance discriminates only when it sits between two measured things — the margin’s threshold has a noise floor below it and a shape floor above it, and both were measured before the threshold was chosen.

Which means it depends on how well the map was measured #

The two curves meet, and where they meet is the answer to a question a reader can act on.

The lower curve’s flatness is the useful half. An impostor’s residual does not know about the measurement at all — it is the geometric difference between two projections over that extent, about one per cent of the map’s width, and adding a tenth of that in noise changes it by nothing visible. So the rule’s false accept behaviour is stable across four orders of magnitude of measurement quality, and only its recall degrades.

About three parts in a thousand of the map’s width is where it stops. On a map thirty centimetres across that is 0.9 millimetres, and to get every candidate rather than most of them the crossings need to be measured to about a tenth of a millimetre.

A coordinate is a number with a width, and this is the same statement about a picture: the precision a measurement is written to decides what can be concluded from it, and the deciding is quantitative rather than a matter of care.

That is a demanding but entirely achievable specification — it is roughly the accuracy of a good scan and a careful digitising pass, and it is far beyond what reading coordinates off a printed page by eye achieves. So the honest instruction that comes out of this rung is: crossings measured to a tenth of a millimetre let the method name the projection or refuse; measured to a millimetre it begins to guess; measured by eye it always guesses.

What this changes about the earlier rungs #

A map does not say what it is reports a win by 3.5 × 10⁷ and treats it as decisive. It is decisive, and now for a stated reason: that number is a margin, it is six orders of magnitude above the largest margin any impostor achieves, and the threshold separating the two populations has been measured rather than assumed.

Two projections that cannot be told apart measures the extent a map must cover before a pair separates. That is the same question with the truth in the library and the extent too small — the margin collapses because the two candidates’ residuals both fall to the noise. So there are two distinct ways for the method to fail, and the margin detects both: too little extent, and no right answer.

What a careless copy hides is the third and is not detected. An affine fit removes the entire difference between the cylindrical equal-area projections, so with an affine nuisance transformation Gall–Peters and Behrmann give a genuine margin of 1 — the truth is present, and the method still cannot name it. A margin threshold refuses, correctly, and reports no answer where an answer exists. That is the right behaviour and it is worth saying explicitly: the rule refuses when the truth is absent and when the truth is unidentifiable, and it cannot tell those apart.

What was computed, and how #

Each trial draws a map’s graticule crossings from a stated projection over a stated extent, optionally adds a stated position error to every crossing from a fixed seed, and runs the full fit — every candidate, its own parameters searched, the best plane similarity removed.

The hold-out is done by removing the true key from the candidate list, which is the only honest way: leaving it in and ignoring its result would still let its parameter search inform nothing, but would change the ranking’s denominator.

The noise is Gaussian in both coordinates, uncorrelated between crossings, and scaled by the map’s own width so that the ladder is scale-free. It is generated from a fixed seed so that every figure is the same picture on every build.

The assertion has four parts and two of them are refusals. With the truth present the fit must name it, every time. With it held out the fit must name something else — trivially true, and worth asserting because a bug that quietly kept the truth in the list would fail it. A margin threshold must exist that accepts more than eighty per cent of the genuine cases with no false accepts. And the two populations must not overlap at all, which is the strongest form and is what actually holds.

What a refusal is worth #

An identification that can refuse is a different instrument from one that cannot, and the difference is not modest.

Without a refusal, every map in an archive gets a name, and the names are right for the maps whose projections are in the library and wrong for the rest — with nothing distinguishing the two. The error rate is then the fraction of the archive drawn in something the library does not hold, which is unknown and is exactly the quantity somebody would want the tool to estimate.

With a refusal, the same archive comes back as three piles: named with a margin above the threshold, refused, and named with a margin close to it. The second pile is the interesting one, because it is the list of projections the library is missing — and running the tool over an archive becomes a way of discovering what to add to it rather than only a way of labelling what is already known.

Where the model stops #

The library is twelve candidates, and none of these means none of these twelve. A map drawn in a projection the library does not contain would be a hold-out case at every trial, and the rule would correctly refuse — but a rule that refuses is not the same as a rule that names the truth, and enlarging the library is the only way to convert one into the other.

The noise model is independent Gaussian error on each crossing. A real digitised map has correlated error — a scan’s distortion varies smoothly across the sheet, so neighbouring crossings are wrong in the same direction — and correlated error is much worse for this method than independent error of the same size, because it looks like a systematic shape difference rather than like noise. That is the same failure a smoothly varying page transformation produces elsewhere, and it is not measured here.

And the margin is a statistic, not a probability. Converting it into a confidence would need a distribution for the residual under each hypothesis, which would need the noise model to be right, which is the assumption the previous paragraph doubts.

What has been published is the ranking, which is the easy half. Producing a winner from a set of candidates is a sorting problem; deciding whether the winner means anything requires a second population to compare it against, and the second population has to be manufactured by removing the answer — which is a small experiment nobody appears to have run.

Who found it, and when #

The statistic is old and the application is not. The ratio of a second-best fit’s residual to the best is the classical F ratio in disguise — Fisher’s, from the 1920s — and in the specific form used here, where the model with the extra structure is compared against the noise, it is the same reasoning as a lack-of-fit test.

Identifying a projection from a picture is a modern problem, created by the existence of enormous numbers of maps in digital form with no metadata attached. The published approaches all rank candidates and none of them, as far as this collection can determine, publishes a rejection rule — which is what makes the question of whether the method can refuse worth asking rather than looking up.

Why a rejection rule is the harder half #

The absence of a published rejection rule in the identification literature is worth examining, because it is not an oversight so much as a consequence of how the problem is posed.

Ranking is a closed question and rejection is an open one. Given a library of candidates, scoring them and taking the best is well defined, terminates, and always produces an answer. Deciding that none of them is right requires a statement about everything outside the library, which is a different kind of claim and needs a threshold that has to be justified rather than computed.

And the closed version is what the applications ask for. A cataloguer with a map and a list of plausible projections wants to know which one it is, and in the overwhelming majority of cases it is one of them — the map was drawn by somebody working in a tradition, using a construction that tradition had. A method that refuses is answering a question that rarely arises.

Rarely is not never, and the cases where it arises are the interesting ones. A map compiled from several sources, drawn by eye, or constructed by a method nobody recorded is exactly the historically interesting object, and it is the one a ranking method will confidently mislabel — because a ranking method has no way to express none of these.

There is also a cost to the closed version that is easy to miss. A ranking method applied to a map outside its library returns the nearest member with a residual, and the residual is often unremarkable — because a projection drawn by eye or compiled from sources resembles several library members about equally badly. So the output looks like an ordinary identification with an ordinary fit, and nothing marks it as the case the method cannot handle.

Which is the same failure this collection keeps finding in a different guise. A procedure that always returns an answer of the same shape gives a reader no way to distinguish the cases it handles from the cases it does not, and the distinction has to be carried by a separate statement that the procedure was never asked to make.

Which is why the rule is worth having even though it fires seldom. Its value is not in the cases it rejects but in what it does to the cases it accepts: an identification made by a method that could have refused is a stronger claim than one made by a method that could not, and the strength is exactly the thing a catalogue entry should be carrying.

And the fix costs one number. A rejection rule adds a second output to the same computation — the ratio the method already forms, compared against a threshold that has to be defended once — so a method that can refuse is not a different method but the same one with its own uncertainty reported. That is the shape of nearly every improvement this collection proposes, and it is cheap for the same reason each time: the quantity was computed on the way to the answer and thrown away.

The threshold is the only part requiring judgement, and it has to be defended once rather than per map.

A defended threshold is also something a later reader can disagree with explicitly.

Where the ladder goes next #

Four rungs have recovered a projection from a graticule, measured the extent a pair needs before it can be separated, found the class of transformations that makes separation impossible, and now measured when the method should decline to answer at all. Every one of them assumes the map has a graticule on it. Most maps do not — and what can be recovered from a map with no graticule, from the shapes on it alone, is a different inverse problem with a much worse condition number.