The bound on the body the country is on

The design problem for a national grid is one number: the spread of its scale factor over the country. Computing that for every conformal projection a country could have adopted, and comparing them all against Chebyshev’s optimum, produced a table with a caveat written into it — the candidates and the bound were both computed on a sphere, because mixing a spherical bound with ellipsoidal candidates compares two different problems, and what the flattening does to a scale spread over a country was left unquantified.

It is quantifiable, and the answer is different for the two halves of the table. Chebyshev’s criterion is stated for a general surface, so nothing in the theorem needed a sphere; only the solver did.

A conformal map of an ellipsoid is a conformal map of a sphere #

The construction that makes this computable is old and is what every national grid already is.

The Earth is a sphere, and when it is not sets out when the spherical approximation is safe. This is a case where it is safe for one number in a table and not for the one beside it.

Replace the geodetic latitude φ by the conformal latitude χ — the auxiliary latitude defined so that the map φ ↦ χ, taken with the longitude unchanged, is conformal from the ellipsoid to a sphere — and then apply any conformal map of the sphere. A composition of conformal maps is conformal, so the result is conformal on the ellipsoid, exactly, and it is the double projection that the Gauss conformal sphere formulation of a national grid describes — the same auxiliary latitude the transverse Mercator series is built on.

Every scale factor of the composite is the plane map’s own, multiplied by the scale factor of the χ map.

For a named candidate, the shortcut was sound #

Nothing moves by more than 0.7 per cent, and the conic moves the other way. The ranking is unchanged, the gaps between families are unchanged, and every conclusion of the earlier comparison stands exactly as written. The spherical shortcut was sound for the part of the table it was used for, and saying so is half of paying the debt.

For the bound, it was not #

For Britain, adopting the spherically optimal map and using it on the real Earth delivers 1,276 ppm against the 1,061 it advertises — a 22 per cent penalty, and enough to lose the whole margin over an ordinary transverse Mercator, which manages 1,414. A design chosen on the strength of the spherical figure would have been chosen on a number nobody could have delivered — which is the same failure the scale factor was chosen describes from the other end, where a number was picked for a reason and then inherited as a fact.

The reason is worth stating carefully because it is the general result. An optimal map has cancelled its own scale variation as completely as its terms allow. That is what optimal means. So when a second variation is added — the χ map’s few hundred parts per million — there is nothing left for it to cancel against, and it appears almost in full. An ordinary candidate already carries 1,400 ppm of its own, and the extra few hundred partly cancels and partly reinforces, which is why it moves by ten.

It is the same effect the Krüger series’ convergence shows in miniature: once the leading terms have been removed, whatever is left is what is measured, and a correction that was invisible against a large error is the whole of a small one.

The better the map, the more the flattening matters. That is the finding, and it is the opposite of the intuition that a small correction matters least where everything is already small.

Solving it properly is one term in one equation #

The Chebyshev criterion says the optimal conformal map of a region is the one whose scale factor is constant on the boundary. The solver already here fits a harmonic function to make that true: the logarithm of the plane map’s scale is a harmonic function of the chart coordinate, so the fit is linear in the coefficients and is a least-squares problem.

On an ellipsoid the quantity that must be constant on the boundary is the composite’s scale, which is the plane map’s times the χ map’s. Taking logarithms turns the product into a sum, so the ellipsoidal problem is the spherical one with log(cos χ / N cos φ) subtracted from the boundary condition — one term added to one right-hand side, and the same linear solve.

That is the whole modification, and it is why the debt was payable at all rather than needing a different solver. It also means the ellipsoidal bound inherits everything the spherical one had: the same convergence with the number of terms, the same residual measurement on the boundary, the same refusal to be beaten by any named candidate.

Three things the ellipsoidal answers do that the spherical ones did not #

Britain’s optimum is below the spherical bound. 1,043 against 1,061 — the country is easier to map on an ellipsoid than a country of the same shape would be on a sphere. The χ factor rises with latitude, and over a region that extends north–south the projection’s own scale variation partly runs the other way, so the two cancel a little. That is not a general fact: for Japan the ellipsoidal optimum is 1,484 against a spherical 1,428, and the cancellation goes the wrong way.

The penalty is largest for the countries where the bound is tightest. Britain and Japan, whose optima are around a thousand parts per million, pay 22 and 21 per cent. Europe and the United States, at twenty-three thousand, pay 4 and 3. The absolute size of the added variation is similar in all four; what differs is how much room there is to hide it.

And the ordering between countries survives. Britain’s grid is still the tightest design problem in the table and Europe’s still the loosest, by the same factor of twenty-two. Everything the earlier rung concluded about which country can be mapped best is unaffected; what changes is the number any of them can reach.

What a country would do with this #

The practical form of the result is short. A national mapping agency choosing a grid today does not choose an optimal conformal map — it chooses a transverse Mercator with a central meridian and a scale factor, because that is what software, legislation and a century of survey marks understand. For that decision the ellipsoidal correction is 0.7 per cent and can be ignored, which is what the earlier rung’s spherical shortcut amounted to assuming.

The result matters for the comparison. Quoting how far an adopted grid falls short of the theoretical best is a claim about the gap between two numbers, and if one of them is computed on the wrong body the gap is wrong by more than either number is. Britain’s grid is 2,094 ppm against an achievable 1,043, a factor of 2.01; against the spherical bound of 1,061 it is 1.97, and against what the spherical design would actually deliver, 1,276, it is 1.64. The same country, the same grid, three different verdicts on how much was left on the table, and only one of them is about the Earth.

What the three verdicts differ about is not the shape of a grid’s scale curve across its zone — that is the same curve either way, balanced by the same constant — but the spread of it over a country, and which body the spread was computed on.

What was computed, and how #

The conformal latitude is the standard closed form, χ = 2 arctan(tan(π/4 + φ/2) · ((1 − e sin φ)/(1 + e sin φ))^(e/2)) − π/2, evaluated on WGS84.

A candidate on the ellipsoid is built by wrapping a spherical projection so that its forward map takes χ instead of φ and its metric is the ellipsoid’s own — leaving the visibility rule, the limit and the distortion machinery untouched, so the two bodies are measured with exactly the same code.

The check that the wrapper is right is that the χ map’s scale factor comes out the same along the meridian and along the parallel. Those are two independent expressions — dχ/dφ over M, and cos χ over N cos φ — and they agree to 4 × 10⁻¹⁰. A wrapper that applied χ to only one of the two coordinates, or applied it and forgot the metric, would fail that immediately.

The bound is chebyshevMap with an added extraLog term in the boundary condition, and both solutions are then re-measured as composites rather than read off the solver. That matters: the solver reports the plane map’s own spread, and with the extra term in place the plane map is deliberately not the thing being measured — it is constant on the boundary only after the ellipsoidal factor multiplies it. A first reading took the solver’s number and reported an ellipsoidal bound of 1,322 ppm, which was the plane map’s spread and not the map’s.

The assertions require the flattening to move the best candidate at all — a measurement that came back identical would mean χ was not being applied — and require no named candidate to beat the ellipsoidal bound, which is the refusal that says the modified solve is still solving the right problem.

The general shape of the finding #

The result that leaves cartography is about optimisation rather than about maps.

An optimum is more sensitive to a modelling change than an ordinary candidate is, and by a factor that grows as the optimum improves. An ordinary candidate carries a large error of its own, and a small additional one partly reinforces and partly cancels against it; an optimum has driven its own error down to whatever its terms allow, so the additional error arrives on top of nothing and appears almost in full.

That is why the flattening moves the best transverse Mercator by 0.7 per cent and the Chebyshev optimum by 22. It is also why an optimum computed in a simplified model should always be re-scored in the full one before it is compared with anything: the simplification that was harmless for the candidates is the whole of the difference for the winner.

Where the model stops #

The bound is still a series of finite order, ten to twelve terms, and its boundary residual is the honest statement of how close to the true optimum it is. Refining it moves the spherical answer from 1,066 to 1,056 over ten to twenty-four terms, so the figures quoted are good to about one per cent, which is well inside the twenty-two per cent effect.

The regions are boxes and ellipses in the library’s own vocabulary, not the actual coastlines of the countries named. The best grid a country could have had already found the answer to be sensitive to which box a country is declared as — 2,107 ppm over one extent and 923 over another — and that sensitivity is far larger than the ellipsoidal correction measured here. The flattening is now quantified; the extent is still a choice, and it is the bigger number.

And nothing here is about the height of the ground. Every scale factor above is on the ellipsoid, and a real grid distance carries the combined factor as well, which for a country with mountains is comparable to everything discussed here — a grid scaled to the ground is not a map is the version of that argument taken to its conclusion.

There is also a reason nobody would have gone looking. The correction is second order in the flattening, which is the standard argument for ignoring it, and that argument is correct for every number in the comparison except the one it is being applied to. Second order in the flattening is not the same as second order in the answer, and the difference between those two statements is the whole of this rung.

Who found it, and when #

Chebyshev stated the criterion in 1856 and Grave proved it in 1911, both on a general surface rather than on a sphere, so the ellipsoidal case has been within the theorem the whole time. What has been missing is a way of solving it, and the double-projection route — reduce to the sphere through the conformal latitude, solve there — is Gauss’s, from the Hanover survey of the 1820s, and is the reason the conformal latitude has a name at all.

The specific arithmetic done here is elementary once both are in hand. It has probably not been done because a national grid is designed once, by a committee, against a target rather than against a bound — and because the difference it makes, twenty-two per cent of a thousand parts per million, is two hundred parts per million, which is a millimetre in five metres and matters to almost nobody.

Why a second-order term became a first-order effect #

The finding’s shape deserves a section of its own, because the reasoning that hid it is sound reasoning applied to the wrong quantity, and the same mistake is available anywhere a bound is computed.

The argument for dropping the term is unimpeachable in isolation. The correction is second order in the flattening, the flattening is about 1/298, so the correction is of order $10^{-5}$ relative to the quantities it modifies. Nobody would keep it in a scale factor, a coordinate, or a distance, and nobody should.

The quantity here is not one of those. It is a bound — the smallest scale spread any conformal map of the region can achieve — and a bound is a small number by construction. The whole answer is about a thousand parts per million, so a two-hundred-part-per-million correction is twenty-two per cent of it rather than a rounding error.

The general shape is that a difference inherits absolute errors and not relative ones. Two quantities of size $A$, each carrying a correction of size $\varepsilon$, have relative errors $\varepsilon/A$ that are negligible; their difference has size $A - B$, which may be far smaller than $A$, and the same $\varepsilon$ against it is $\varepsilon/(A-B)$. Every optimality gap, every residual, every improvement figure and every bound is such a difference.

The same trap is waiting wherever this collection reports an improvement. This aspect is twelve per cent better than that one is a difference of two distortion figures; the compromise beats its parents by twenty-six per cent is a difference of three; every optimality gap on the site has the property that the interesting number is much smaller than the numbers it was computed from. In each case a term dropped as negligible against the ingredients has to be re-examined against the result, and the re-examination is arithmetic rather than judgement: divide the dropped term by the reported gap rather than by the inputs.

What makes it hard to remember is that the dropping usually happens in a different piece of code, written for a different purpose, by somebody who was right at the time.

So the rule to carry is about where a term is dropped rather than how small it is. Order-of-magnitude reasoning about a correction has to be conducted against the quantity finally reported, not against the intermediate quantities the correction appears in — and when the reported quantity is a gap between two nearly equal things, almost nothing may be droppable.

That is why the shortcut was sound for a named candidate and unsound for the bound, and the two uses sit in the same essay with the same arithmetic behind them. Nothing about the flattening changed between them; what changed is what the answer was a difference of.

Where the ladder goes next #

Seven rungs have built a grid, measured what it costs, designed one for a region, compared two over the same ground, and now solved for the best one on the right body. Every one of them treats the country as a fixed extent known in advance. It is not: a grid outlives the borders it was designed for, and what happens to an optimal design when the region it was optimal for changes shape is the question a century-old national grid actually answers.