Report the map, not the parameters

The previous rung mapped the objective the aspect search walks on and found it lumpy: twenty-six to thirty-four local minima on a coarse cube, and a downhill walk from a random start reaching the best of them between six and thirty-five per cent of the time. Seeding from the grid’s own lowest points fixed the search, and the essay ended by recording a shortfall it could not pay.

The shortfall was this. Run the same search at three grid resolutions and it returns the same score to 2.3 per cent from poles sixty degrees of latitude apart. The score is reproducible and the parameters are not, and what a reader is given is the parameters.

That was recorded as a problem. It is worth asking, before trying to fix it, whether it is one — because a triple is not a map, and the question a reader actually has is about the map.

The comparison that was missing #

An aspect is a rotation of the sphere, given by three numbers: the longitude and latitude of the rotated pole and a third angle about it. What it produces is a map, and what the map does to a region is a field: a point scale and an angular deformation at every point in it.

Comparing two aspects on their triples is comparing two labels. Comparing two aspects on their fields is comparing two maps, and it is a comparison nothing in this ladder had made, because until the third parameter was run there was no reason to think two different triples could produce the same map.

The measurement is direct: sample the region, evaluate the areal factor and the angular deformation under each aspect at each sample, and take the largest and root-mean-square differences. It needs no new machinery — the distortion field is what this collection has computed since its first figure, and what survives a change of coordinates is exactly the part of it that a comparison between two aspects may use.

The scale it has to be read against is the region’s own deformation. A difference of a quarter of a degree means nothing until it is set beside what the map is already doing, which for this projection over this region averages 1.22° and reaches 2.05°.

The three answers are one map #

Moving the pole deliberately away from the optimum prices the trough. Twenty-eight degrees of movement costs 10.9° of angular deformation, and the band the search’s own three answers sit in is 0.29° wide while their poles differ by 64.9° — so the three sit inside a trough two orders of magnitude shallower than a deliberate mistake.

The second half of that figure is what makes the first half evidence. If every pair of aspects produced nearly the same field, agreement would mean nothing. A pole moved twenty-eight degrees produces a map that differs by 10.9° RMS — nine times the region’s own mean deformation, and thirty-seven times the disagreement between the search’s answers.

So the three searches agree about the map to within a quarter of what the map itself does, and disagree about the pole by two thirds of a right angle. The parameterisation is the thing that is not reproducible.

Two mechanisms produce that, and separating them matters.

The first is an exact degeneracy. A pole and its antipode with the third angle reversed are the same rotation, written two ways. The search has no reason to prefer either, and at one of the resolutions tried on a European region it returned (94.1°, 42.3°, γ = −128.9°) where another returned (−86.0°, −42.2°, γ = 128.9°) — the antipodal pair, to a tenth of a degree, with identical scores. That is not a failure of the search. It is a two-to-one parameterisation, and it can be removed by a convention.

The second is a genuine valley. Away from the exact degeneracy the objective has a curve along which the score changes very little and the map changes very little, and the search stops wherever its grid happened to put it. That cannot be removed by a convention, and it is the interesting case: the objective really does not care, over a range of a hemisphere, because the region is small enough that most of the rotation is spent moving parts of the sphere the region is not on — which is why an aspect fitted to a region is fitted to so little of the sphere.

A convention that removes half the problem #

The antipodal degeneracy can be removed and it costs one line, so it is worth writing down what the convention has to be.

A rotation of the sphere taking the north pole to (λ, φ) and then turning by γ about the new pole is the same rotation as one taking it to (λ + 180°, −φ) and turning by γ ± 180°. So every aspect has exactly two names in this parameterisation, and choosing between them is a matter of picking a half-space.

The natural choice is the pole in the northern hemisphere, with the equatorial case broken by requiring the longitude in [0°, 180°). Applied to the European result above, (94.1°, 42.3°, −128.9°) is canonical and (−86.0°, −42.2°, 128.9°) is not; applied to the Japanese results, the three answers become (5.6°, 0.5°, 43.9°), (49.4°, 60.3°, 70.2°) and (59.8°, 44.4°, 68.2°), which are still 64.9° apart. The convention removes an artefact and leaves the real spread untouched, which is what a good convention does.

There is a second convention worth having and it is about the reader rather than the search. An aspect is far easier to check when it is reported as the point of the region that ends up at the map’s own centre, which is one longitude and one latitude on the ground rather than a pole in a rotated frame, plus the rotation about it. That is the same three numbers rearranged and it is the arrangement in which two answers differing by 64.9° of pole become two answers whose region centres differ by very little — because it is the region’s own position that both searches agreed about.

What to publish instead #

If the triple is not reproducible and the map is, then the triple is not the answer. Three things can be published and each of them has a different cost.

The map itself, as a distortion summary: the region’s worst and mean angular deformation, its scale range, and the position of the worst point. That is reproducible to the accuracy of the field agreement — a quarter of a degree here — it is what a reader wants to know — the answer to which projection is best always names a purpose — and it does not let anybody reproduce the map.

A canonical triple, obtained by applying a tie-break to the two-to-one degeneracy — fold every pole into the northern hemisphere and negate γ when it moves — and then reporting whatever the search found. That is cheap and it fixes the antipodal half of the problem entirely. It does nothing for the valley.

The search, meaning the objective, the region samples and the grid — enough that a reader can re-run it. That is what a reproducibility-minded practice would do and it is not what a map’s metadata has room for.

The practical recommendation this rung makes is the first plus the second: publish the distortion summary as the result and the canonical triple as the recipe, and say which grid the triple came off. A reader who re-runs on a different grid will get a different triple and the same summary, and will now be able to tell that this is agreement rather than disagreement.

A trough that shallow is only meaningless if the depth is small against something, and the something is the distortion the projection imposes before any aspect is chosen at all.

The scale of the comparison is what settles it. The angular deformation the chosen projection already imposes over the region runs from 0.19° to 2.05° and averages 1.22°, so a difference of 0.29° between two candidate aspects is a quarter of that and a difference of 10.9° from a pole moved twenty-eight degrees is nine times it. Neither number means anything until it is put beside the third.

What was computed, and how #

The searches are the three-parameter ones from the previous rung, run at 12 × 6 × 8, 18 × 9 × 12 and 24 × 12 × 16 grid points with a compass walk from each of the best few seeds and from the two-parameter answer. Nothing about the search is changed; the only new thing is the comparison afterwards.

The field distance samples the region on its own grid, evaluates both aspects at each sample, and skips any point one of them cannot show — which for a projection with a limit is a real consideration and for Robinson is none. It reports both the RMS and the worst departure, because a summary that agrees on average and disagrees badly at one point is not the same claim.

The whole run takes about six seconds and is memoised, because the three figures and the assertion all want the same answer and a search that is re-run per figure is a search whose numbers can drift between figures on the same page.

The shortfall, and what it turned out to be #

It is worth closing the loop on the shortfall explicitly, because the way it was wrong is more useful than the way it was right.

What was recorded was: the three-parameter aspect search’s parameters are not reproducible; what is owed is a way of reporting an aspect that a reader can check. Both halves of that are correct. The unstated assumption underneath was that non-reproducible parameters mean a non-reproducible answer, and that assumption is what this rung tests and rejects.

The debt is therefore paid in a different currency than the one it was denominated in. There is no new search, no better seeding and no tighter convergence criterion — the search was already fine, and the previous rung had already shown that seeding from the grid’s own lowest points reaches the best minimum in one attempt every time. What is new is a second way of comparing two answers, and a demonstration that on that comparison they agree.

The general lesson for a recorded shortfall is that it should name the symptom and not the diagnosis. This one named both, and the diagnosis was wrong while the symptom was real.

Where the model stops #

One projection and one region. Robinson over Japan is a compromise projection over a small mid-latitude region, which is the case where the valley is widest — a small region leaves most of the rotation unconstrained. A hemispheric region pins the aspect much harder and its parameters will be far more reproducible; a long thin region pins two of the three parameters and leaves the third loose. The phenomenon is general and its magnitude is not.

The field distance is a summary of a summary. Two maps agreeing to 0.29° RMS in angular deformation could still differ in some way the summary does not carry — in where the worst point is, or in the direction of the deformation rather than its size. The worst-point departure is reported alongside and is 0.61°, which is reassuring rather than conclusive.

The criterion is fixed. Every comparison here uses the Kavrayskiy mean-square logarithmic scale error, which is one of several the ladder has already shown disagree with each other. A different criterion has a different objective, a different optimum and a different valley, and whether its answers agree on the map as well as these do is a separate measurement.

And the degeneracy is only partly characterised. The antipodal one is exact and is identified. Whether the valley is a curve, a surface, or a set of disconnected basins that happen to score alike is not established here, and it is the question that would decide whether a canonical form exists at all. It is a fractured sheet: one connected piece at a loose threshold, fourteen disconnected basins once the threshold passes twice the optimum.

The generalisation #

A quantity that is not reproducible is not automatically a problem; the question is whether it is the quantity anybody uses. The shortfall this rung pays was recorded in good faith and it was recorded about the wrong object, because the object that was easy to compare was the one that was printed.

The general form is gauge freedom, and this collection has just met it from the other end: a network’s coordinates depend on the datum and its residuals do not. The same discipline applies here. Report what is invariant; report the convention separately; and do not treat a change in the convention-dependent part as a change in the answer.

The failure mode this guards against is the one where a reproducibility check is run on the wrong quantity and reports a crisis. A clustering that returns different labels on different runs, a factorisation that returns different signs, an optimiser that stops at a different point of a flat valley — each of them can be reported as instability, and each of them is stable in the quantity that matters. The check has to be applied to the thing the reader consumes, and finding out what that is takes longer than writing the check.

Who found it, and when #

The idea of choosing an aspect by minimising a distortion measure over a region is Nicolas Tissot’s in principle and Vladimir Kavrayskiy’s in practice; the criterion this collection uses is the mean-square logarithmic scale error that carries Kavrayskiy’s name from the 1930s. Chebyshev’s much earlier criterion — that the best conformal map of a region has constant scale on its boundary — is the one result in the area that gives a characterisation rather than a search, and it is why Chebyshev’s criterion is the ladder’s third rung.

The general problem of a fitted parameter being unidentifiable while the fitted object is well determined is old in statistics and has a vocabulary — identifiability, estimability, the difference between a parameter and an estimable function. Geodesy has its own version in the free-network adjustment. Neither vocabulary seems to have reached the projection-selection literature, where an optimal aspect is reported as three numbers and the question of whether those numbers are determined is not usually raised.

The antipodal degeneracy is elementary and is presumably known to everybody who has implemented such a search; it does not appear to be written down, which is what happens to facts that are obvious the moment somebody hits them.

The publishable form: a checksum of the map #

The rule report the map, not the parameters is easy to agree with and awkward to obey, because a map is not a number and a paper wants numbers. There is a form that is both, and it costs nothing.

Publish a hash of the projected coordinates of a fixed sample of places. Take a stated set of test points — a few hundred, on a stated lattice, in a stated order — run them through the fitted projection, round to a stated precision, and hash the result. That single string is a checksum of the map itself, and it has the two properties the parameters do not.

It is invariant under the degeneracies. Two antipodal parameter triples that produce the same map produce the same coordinates and therefore the same hash. A search that lands anywhere in a flat valley reproduces the hash of every other point in that valley, which is the honest statement that the valley is one map.

And it is sensitive to what actually differs. Two implementations agreeing on all three parameters but differing in a convention — the sign of a rotation, the order in which two of them are applied, degrees against radians somewhere in the middle — produce different coordinates and different hashes. That is the failure a parameter table cannot catch and is exactly the failure that reproduction attempts run into.

The cost is a paragraph of specification and a few hundred function calls. What has to be stated is the sample, the ordering, the rounding and the hash, and none of those is a research decision. What is gained is a claim a second party can refute in one run, which is the only kind of claim this collection is interested in making.

It is worth being explicit that this does not replace the parameters. They remain the right thing to report for anybody who wants to understand or vary the map. The hash is what settles whether two people have the same one, and those are different jobs that a table of three numbers has been quietly doing both of.

Where the ladder goes next #

This rung finds a reported quantity that was not the reproducible one. The bodies ladder has a debt with the same shape: it solved a conformal map on a triangulated surface and reported an areal spread, and never asked whether that number was a property of the body or of the two vertices the solve happened to pin.