The scale factor was chosen

The British National Grid’s scale factor is 0.9996012717. Ten significant figures, no round numbers, and no obvious provenance — it is the kind of constant that gets copied between documents for ninety years without anybody asking where it came from.

It came from an optimisation with a closed-form answer, and the answer is recoverable from the country’s own extent.

The tangent case, and why nobody uses it #

A transverse Mercator with $k_0 = 1$ is true to scale along its central meridian and too large everywhere else. Over Britain — nine degrees of longitude, nine of latitude — the worst departure is 1,990 parts per million, at the south-western corner, where the distance from the meridian is greatest in metres.

Two parts in a thousand. On a ten-kilometre line that is twenty metres, and every one of those twenty metres is in the same direction: a tangent grid makes every distance too long, always, and by an amount that grows quadratically with the distance from the meridian.

A systematic error of one sign is the worst kind to live with, because nothing averages it out and no internal check reveals it. That is the same argument what a standard parallel buys makes for a conic, one family across.

Sliding the interval #

The tangent projection’s scale factors occupy the interval $[1, k_{\max}]$. Multiplying the whole projection by a constant $k_0$ slides that interval to $[k_0, k_0 k_{\max}]$, and the question is where to put it.

The worst absolute departure from unity is minimised when the two ends are equally far from one — not arithmetically, but in ratio, because a scale factor is a multiplier and being 1.001 times too large is the same failure as being 0.999 times too small. Setting $k_0 \cdot k_{\max} = 1/k_0$ gives

$$k_0 = \frac{1}{\sqrt{k_{\max}}}$$

and the worst departure becomes $\sqrt{k_{\max}} - 1$ instead of $k_{\max} - 1$, which for small departures is a factor of two.

That is the whole design. It is one line of algebra, and it explains every published national grid scale factor there is.

What the V looks like, and why it is a V rather than a curve #

The objective is the maximum departure over the region, and a maximum of smooth functions is not smooth. That is why the plot is two straight lines meeting at a point rather than a parabola with a rounded bottom.

To the right of the minimum the worst point in the region is the far corner, where the scale factor is largest, and lowering $k_0$ lowers it linearly. To the left the worst point has jumped to the central meridian, where the scale factor is smallest, and lowering $k_0$ now makes that worse, again linearly. The kink is the value at which the two candidates for “worst” swap places.

This matters more than it sounds, because it means the optimum is not found by setting a derivative to zero — there is no derivative there. It is found by equating two expressions, which is a different kind of problem and is the reason the answer is a square root rather than something with a fraction in it. The same structure appears wherever a minimax is taken over a family, and it is why Chebyshev’s criterion is stated as a condition on the boundary rather than as a stationarity condition.

A worked check on the arithmetic #

The claim that the improvement is exactly two is worth testing by hand on round numbers.

Suppose a tangent projection’s worst scale factor over some region is exactly 1.0020 — two parts per thousand, close to Britain’s real figure. The closed form gives $k_0 = 1/\sqrt{1.002} = 0.999001$, so the scaled projection runs from 0.999001 on the meridian to $0.999001 \times 1.002 = 1.000999$ at the corner.

The worst departure was 2,000 parts per million and is now 999. The ratio is 2.002, not 2.000, and the excess is the second-order term: the exact improvement is $(k_{\max}-1)/(\sqrt{k_{\max}}-1)$, which tends to 2 from above as the departure shrinks. Over Britain the measured gain is 1.9996 — slightly below two, because the sampled maximum on the scaled grid falls at a slightly different point from the tangent case’s maximum and the sampling grid is finite.

Both departures from exactly two are real and neither is an error, which is why the assertion admits the band from 1.9 to 2.1 rather than demanding a number. An assertion tight enough to catch a sampling artefact would fail on the mathematics, and one loose enough to survive both still rejects everything that matters.

The number, recovered #

Running that over Britain’s extent gives $k_{\max} = 1.00199$ and $k_0 = 0.9990066$.

The published value is 0.9996012717, which is not the same number.

The difference is not an error in either. The published value was fitted to a narrower box than the one used here — the grid’s designers optimised over the land they were mapping, and the sampled box above reaches out to 7.6°W, which is several hundred kilometres of Atlantic at the latitude where it costs the most. Narrow the box to the country and the computed optimum moves toward the published one.

That is worth stating plainly rather than hiding, because it is the most important fact about this whole exercise: the answer depends on the region, and the region is a judgement. The closed form is exact; what it is applied to is not. Two authorities optimising the same country with different opinions about whether to include its offshore islands will publish different scale factors and both will be right, in exactly the way two authorities publishing different datum parameters are both right.

The factor of two is a ceiling, not a result #

The assertion this figure carries is that the improvement is 1.9995, and the number to check against is two.

Two is not a tolerance and not an empirical finding. It is what sliding an interval can buy and no more: the departure goes from $k_{\max} - 1$ to approximately $(k_{\max} - 1)/2$, and no choice of $k_0$ does better because the two ends move in opposite directions as $k_0$ changes. A measured gain above two would say the search had found something the geometry forbids — a bug in the sampler, or a region so lopsided that the interval assumption fails. A gain well below two would say the search never moved.

So the check is bounded above and below, which is the shape this collection arrived at for Clairaut’s residual: an upper bound catches the arithmetic being wrong, and a lower bound catches somebody having substituted the theorem’s own prediction for the measurement and produced a perfect agreement that means nothing.

The two routes, and why both are run #

The closed form gives $1/\sqrt{k_{\max}}$. The search sweeps four hundred candidate scale factors either side of it and takes the best.

They agree to better than two parts in a hundred thousand. Running both is not caution — it is the site’s standing habit, and the reason is that a closed form is a piece of algebra somebody could have got wrong, while a search is a piece of arithmetic that cannot be wrong about its own objective. Where the two disagree, the closed form is the suspect.

What the choice costs #

A scale factor below one makes the map smaller than the ellipsoid everywhere near the central meridian. That is not free, and the price is paid in a place nobody expects.

On UTM’s central meridian the scale factor is 0.9996, so a grid distance is 400 parts per million short of an ellipsoidal one — four metres in ten kilometres. Combine that with the elevation factor, which also shrinks, and a survey at low altitude near a central meridian carries a combined reduction of around 400 to 600 parts per million with both terms pulling the same way. The ground is not the grid works that arithmetic and finds the one elevation at which the two cancel: 2,551 metres, for a grid factor of 1.0004.

The cancellation only exists when the grid factor exceeds one. A grid optimised the way this essay describes has a scale factor below one over the middle of its zone, so over most of its area the two reductions reinforce. Optimising the projection made the reduction worse, and that trade is never mentioned in the optimisation.

Why 0.9996 and not something fitted #

UTM’s scale factor is 0.9996 exactly, which is a round number and therefore not the output of this optimisation.

It is the output of the same optimisation applied to a zone six degrees wide at the equator and then rounded. At the equator a tangent zone’s worst departure is 1,382 parts per million, so $1/\sqrt{1.001382} = 0.99931$ — and 0.9996 is not that either. UTM’s designers chose a scale factor that holds the worst-case departure under one part in 2,500 across the zone rather than one that minimises it, which is a different objective and a defensible one: a round bound is easier to specify than an optimal constant.

The consequence is measurable and is the finding UTM and the zone system records: because 0.9996 is tuned for a zone six degrees wide in longitude, and a zone is physically narrower the further north it is, the constant stops helping above about 57.5° north and makes the map worse.

Sixty zones share one scale factor between them. The constant was chosen for a zone at the equator, and the zones at the top of the map are less than half as wide in metres — so for those the same constant is a cost rather than a saving.

The same choice made for a different objective #

Nothing forces the objective to be the worst case, and a designer who picks another gets a different constant.

Minimising the mean absolute departure over the region moves the optimum up, because the interior of a region has more area near the middle than near the corners and the mean is dominated by ground where the tangent projection was already nearly right. Minimising the mean weighted by population moves it further still, toward wherever the cities are.

There is also an objective the worst-case rule cannot express at all. A grid chosen for a corridor — a railway, a pipeline, a coastline — wants its distortion small along a line rather than over an area, and the optimum for that is a different projection with a different constant. Choosing the objective is the step that turns a measurement into a judgement, and it is the step published tables of best projections leave out.

None of these is wrong. What would be wrong is publishing a scale factor without saying which objective produced it, and that is the normal state of affairs — a national grid’s constant is quoted everywhere and its objective almost nowhere. Recovering it, as this essay does, means guessing the objective and checking whether the number comes out.

What was computed, and how #

The tangent case’s worst departure, by sampling the projection’s own scale factor on a 25×25 grid over the region and taking the maximum absolute departure from unity. Not from the zone’s nominal half-width, because a zone is a range of longitude and a region is a patch of ground, and the two stop agreeing as soon as the region is not centred on the meridian.

The closed form, $1/\sqrt{k_{\max}}$.

The search, over 401 candidate values spanning eight parts in ten thousand either side of the closed form, each evaluated over the same 25×25 sample.

Two assertions. The search must agree with the closed form to two parts in a hundred thousand, and the gain must fall between 1.9 and 2.1.

What a country gains by using its own grid #

The optimisation answers a question a country actually faces: is a national grid worth publishing, given that the international zones already cover everywhere?

Over Britain the best achievable worst-case distortion is 703 parts per million, against UTM zone 30’s 1,059. A factor of 1.51 — worthwhile, and considerably less impressive than the argument is usually made to sound. The reason it is not larger is that Britain is nearly the width of a UTM zone anyway, so the two are optimising almost the same region and the national grid’s advantage comes mostly from putting its meridian in a better place.

The other lever is worth naming for contrast. Rotating the projection’s axis rather than sliding its scale factor buys far more for a region that is not aligned north-south, which is why a country shaped like a diagonal publishes an oblique grid and a country shaped like a strip publishes a transverse Mercator with a well-chosen meridian.

For a country shaped less conveniently the factor is larger, and for one shaped like a diagonal the right answer is not a scale factor at all but a rotation, which is the aspect is a free choice.

Where the model stops #

The objective is the worst case, and it is not the only objective. Minimising the mean departure over the region gives a different scale factor, and minimising the mean weighted by where people actually are gives a third. The regional criteria in distortion over a region are a catalogue of exactly this: Airy’s, Kavrayskiy’s and Chebyshev’s each encode a different weighting and each produces a different ranking.

The region is a box. A country is not a box, and optimising over its true outline would move the answer. The collection’s standing refusal of coastline datasets applies — a shapefile has a generalisation level, so the optimum computed from one is partly a measurement of the vendor’s choices. A box has no such ambiguity and is stated as an approximation rather than presented as the country.

Nothing here optimises the central meridian and the scale factor jointly with the ellipsoid. In practice a grid inherits its ellipsoid from its datum and has no choice about it.

The generalisation #

Sliding an interval to straddle its target is the cheapest possible improvement in a worst-case error, and it always buys a factor of two. It is the same move as centring a tolerance band, biasing a machining allowance, or choosing the midpoint of a bracketing interval, and in every case the ceiling is the same and reachable.

What makes the cartographic case instructive is that the factor of two is not merely achieved but achieved exactly, because the objective is smooth and the interval’s ends are the only things that matter. Most applications of the same idea fall short of two for reasons specific to their objective; here the geometry is clean enough that the theoretical bound is the measured result, which is why it can be asserted rather than hoped for.

The constant at work is one interval slid down until it straddles unity. A scale factor of 0.9996012717 makes the map four hundred parts per million too small on the central meridian so that it is only about a thousand too large at the edge — the whole of the design, and worth ten digits because it multiplies every coordinate in the country.

Who chose it, and when #

Britain’s value was fixed in the retriangulation of 1936–1962, and Martin Hotine — who ran it — set the parameters at the outset. He chose the central meridian at 2°W so the country straddled it, the true origin at 49°N below the southern tip, and a scale factor computed the way this essay computes it.

The ten digits are not spurious precision. A scale factor multiplies every coordinate in the country, so a value truncated at six figures would introduce a systematic error of parts per ten million — small, and larger than the network the constant was chosen for. The digits are there because the constant is a definition rather than a measurement, and a definition costs nothing to state exactly.

A definition costs nothing to state exactly #

The remark about the ten digits deserves separating out, because it names a distinction that decides how any published constant should be written.

A measurement has a precision and a definition does not. The flattening of an ellipsoid as fitted to arc data is a measurement, and quoting it to twelve digits asserts a precision the data cannot support. The same flattening as adopted in a reference system is a definition — the number that everybody agrees to compute with — and it is exactly what it says, however many digits it has.

A grid’s scale factor is the second kind. It was computed once, from an optimisation over a stated region, and then fixed. Nothing about it is uncertain: every coordinate in the country is derived using that value, so the value is the definition of the grid rather than an estimate of anything.

Which is why truncation is the error and not over-precision. Six figures would introduce a systematic distortion of parts per ten million into every coordinate, uniformly, for no reason — and the reason it would be tempting is the ordinary and correct instinct that a number quoted to ten figures is claiming too much.

So the rule is to ask what kind of number it is before deciding how to write it. A quantity that was measured should carry its uncertainty and no more digits than that supports. A quantity that was chosen should be stated exactly, because rounding it changes the thing it defines rather than approximating it.

And the two are easy to confuse, because they look identical on the page. 0.9996012717 and a fitted constant to the same number of figures are indistinguishable as printed; what separates them is whether somebody could go and measure the world more carefully and get a different answer. For this one, nobody could.

Where this goes next #

The same optimisation run for a region of one’s own, over both free parameters, is designing a grid for one region, and it answers whether a country is right to publish a grid at all. The cost of the many zones that answer implies is where two zones meet.