A datum is fitted to a region

Airy 1830 for Britain. Bessel 1841 for central Europe. Clarke 1866 for North America. Everest 1830 for India. Hayford’s International Ellipsoid of 1924 for a continent that could not agree. Forty-odd ellipsoids, all of them purporting to describe the same planet, differing by hundreds of metres in the equatorial radius.

The usual explanation is that measurement was bad and each attempt was a better guess. That explanation is wrong, and it obscures the more interesting thing: they were not competing estimates of one quantity. Each was fitted to a different piece of ground, for a purpose that made the rest of the planet irrelevant, and each was better at its job than any global figure of its day.

The claim #

A survey does not measure the distance to the centre of the Earth. It measures angles between marks, and one baseline, and it reduces the result onto a reference surface. What it is therefore sensitive to is the curvature of that surface — how quickly the vertical turns as an instrument is carried across the ground — and not to the surface’s absolute size.

So the question a national geodesist actually faced was: what figure has the right curvature here? And the answer to that question is local by construction, because the curvature of the Earth is not one number — it varies by 0.7 per cent from equator to pole, and by measurable amounts across a country.

The claim of this essay is that fitting locally buys a specific and computable thing: it removes a systematic bias, and it leaves a spread that no sphere and no ellipsoid can touch. The two are different in kind, and confusing them is what makes the forty ellipsoids look like forty errors.

What was computed, and how #

The Gaussian radius of curvature at latitude $\varphi$ is the geometric mean of the two principal radii:

$$R = \sqrt{MN}, \qquad M = \frac{a(1-e^2)}{(1 - e^2\sin^2\varphi)^{3/2}}, \qquad N = \frac{a}{\sqrt{1 - e^2\sin^2\varphi}}$$

$M$ is the radius in the meridian, $N$ in the prime vertical. Their geometric mean is the radius of the sphere that has the same total curvature at that point, which is the isotropic choice — the one that does not privilege a direction — and it is what the standard reduction of a survey uses.

The measurement is then arithmetic. Sample $R$ across a band of latitude, and compare two candidate spheres against it: the global mean radius, 6,371.0 kilometres, and the mean of $R$ over the band itself. For each, report two numbers separately:

• the bias, the mean signed departure, in parts per million;
• the spread, the root-mean-square departure, in parts per million.

Over ten degrees of latitude centred on 54.5° north, the global radius carries a bias of 2,203 parts per million and a spread of 2,227. The locally fitted radius carries a bias of exactly zero — by construction, since it is the mean — and a spread of 325.

The gain is a factor of 6.9. Not a factor of a thousand, which is what “fitted to Britain” sounds like it should mean, and not a factor of one either.

Why the gain is what it is, and not more #

The 325 parts per million that survive the local fit are not slop. They are the curvature varying across the band, and a sphere has one curvature everywhere.

That is the honest ceiling on what any local sphere can do, and it is the reason the national figures were ellipsoids and not spheres: an ellipsoid has a curvature that varies with latitude, so it can follow the variation as well as the level. What a national ellipsoid could not follow was the variation that is not a function of latitude — the departure of the actual figure from any ellipsoid of revolution, which is the geoid, and which height above what? takes up.

Those two figures are the same generator with different arguments, and the pair is the argument. Over ten degrees the local fit is worth a factor of seven; over the whole Earth it is worth nine per cent. The trade has the shape every trade in this collection has: the narrower the purpose, the larger the gain, and the less transferable the result.

The bias is what actually hurt #

Two metres in every kilometre is worth dwelling on, because it is the number that made national ellipsoids necessary rather than merely nice.

A bias in the reference radius is a scale error, and a scale error is the one kind of error that does not average out. Random error in a triangulation shrinks as the network grows, because independent errors partially cancel. A systematic scale error grows in proportion to the network: measure a baseline in Kent, propagate the triangulation to Scotland, and a 2,200 ppm error in the assumed radius accumulates to about 1.8 kilometres over 800 kilometres of chain.

That is not a subtle effect and it is not a rounding. It is the difference between a survey that closes and a survey that does not, on instruments that could measure an angle to a second of arc.

The same logic runs through the grid, one layer up. What a standard parallel buys and UTM and the zone system are both about converting a systematic error into a smaller error of both signs, and the scale factor of 0.9996 does at the level of the projection exactly what a locally fitted radius does at the level of the datum.

Fitting two parameters instead of one #

The one-parameter version above is the argument in its clearest form and it is not what Airy or Clarke actually did. An ellipsoid has two shape parameters, and both were fitted, from measurements of the length of a degree of latitude at two or more widely separated places.

That amplification is the reason the national figures disagree as much as they do. The equatorial radius comes out of the inversion robustly; the flattening does not, because it is carried entirely by the difference between two arcs that are each about 111 kilometres long and differ by a kilometre. An arc measured to a hundred metres — which is respectable for a chain of triangles across a continent in 1840 — fixes the flattening to about ten per cent.

So the spread among the nineteenth-century ellipsoids is mostly a spread in flattening, and mostly conditioning rather than carelessness. The figure of the Earth was measured follows that inversion all the way to the point where it returns a negative answer.

What a badly fitted figure costs a coordinate #

Two hundred metres is not the fitting error. A figure fitted to Britain reproduces Britain’s curvature to a few hundred parts per million, which over the country is metres rather than hundreds of metres. The two hundred metres is the placement — the third of the four commitments in what a coordinate refers to — and it is free precisely because the survey is insensitive to it.

That is worth stating plainly because it is counter-intuitive: the quantity a survey could measure well was fitted well, and the quantity it could not measure at all was left wherever it fell. Nobody was careless. The unmeasured degree of freedom went unmeasured.

What was traded away #

The forty ellipsoids were a good answer to the question each of them was asked, and they produced a problem nobody had anticipated: the answers do not agree with each other, and the disagreement is not resolvable by measuring better.

Two neighbouring countries on different datums have a border that is in two places. Every joint survey has to declare a conversion. Every chart that crosses a boundary carries an inconsistency the size of a hundred metres. And each of the two figures is correct on its own ground, so the disagreement has no winner.

The global figure is what buys that back, and what it costs is the two metres per kilometre — which is affordable now precisely because a modern survey does not propagate scale through a triangulation chain. It measures distances directly, against a satellite constellation whose orbits are computed in a geocentric frame. The instrument changed, and the answer to “what should the reference figure be” changed with it.

That is a general shape worth naming: the right compromise depends on the instrument, and it moves when the instrument does. Nothing about the Earth changed between 1830 and 1984.

The one latitude at which local fitting buys nothing #

The gain is a ratio of a bias to a spread, and the bias is the distance from the band’s own mean curvature to the global one — so it depends on where the band is, not only on how wide it is. That has a consequence the three examples above do not show, because all three are read as a story about size.

The Gaussian radius runs from 6,356.75 kilometres at the equator to 6,399.59 at the pole, and the conventional mean radius of 6,371.0088 sits inside that range. It is attained at 35.32° of latitude. A band centred there has almost no bias for a local fit to remove, so the local figure and the global one are equally good over it and the gain collapses towards one whatever the band’s width.

The gain therefore has a zero, and the zero is at a latitude decided by an arithmetic convention — the mean radius is defined as (2a + b)/3, and a different but equally defensible mean would put the crossing somewhere else. Nothing about the Earth distinguishes 35.32°.

So the trade has two variables and the essay’s three examples move only one of them. Britain at 54.5° is far from the crossing and gains a factor of seven; a country the same size centred on Cyprus or northern Algeria would gain very little, from the same argument, with the same instruments and for a reason no geodesist chose. A national ellipsoid was worth more to some countries than to others, and how much was decided by latitude rather than by care.

Where the model stops #

Three limits, stated rather than skirted.

This measures the fit to an ellipsoid, not to the ground. The curve in the hero figure is WGS84’s own curvature, so what is being fitted is one mathematical surface to another. The actual figure of the Earth departs from any ellipsoid by up to a hundred metres — the geoid — and a national datum’s fit to its own country was a fit to that, not to WGS84. What is demonstrated here is the mechanism and its size, using a surface whose properties are exactly computable; the historical fits were to something with no closed form.

A one-parameter fit is not what the historical fits did. Fitting only a radius is the simplest version of the argument. Airy, Bessel and Clarke each fitted two parameters, $a$ and $f$, to a set of arc measurements — which is the inversion that turns out to be badly conditioned enough to explain a great deal of the spread between their answers.

The bias here is a bias in curvature, not in position. A datum’s placement — where the ellipsoid’s centre sits — is a separate commitment and a larger effect, and it is the seven parameters that carry it. A country could and did adopt a well-chosen ellipsoid and then place it hundreds of metres from the geocentre, because the placement was not measurable and did not matter for its purpose.

The generalisation #

The distinction this essay turns on — a removed bias against an irreducible spread — is worth carrying out of geodesy, because it decides whether “fit it locally” is a good idea.

Local fitting removes whatever the model can represent and leaves whatever it cannot. If the residual is dominated by a constant offset, fitting locally is transformative: the offset goes entirely. If the residual is dominated by variation within the region, fitting locally does almost nothing, because the model has no freedom left to spend on it.

So the useful question before fitting anything to a subset is not “will this be better here” — it will — but what fraction of the error is bias and what fraction is spread. Over ten degrees of latitude the answer is 87 per cent bias, so the local fit is worth having. Over the whole Earth the answer is almost all spread, so it is not.

And the cost is always the same and is always paid later: a model fitted locally does not compose. Two of them cannot be used together without a conversion, the conversion is a fit in its own right, and the conversion has residuals — which is the entire subject of where a fit leaves residuals.

The reduction a surveyor actually applies #

The abstraction above has a very concrete form on a job sheet, and it is worth following once because it shows where the fitted radius enters.

A distance measured between two marks on the ground is not the distance between them on the reference surface. It has to come down — divided by $(R + h)/R$, where $h$ is the height above the ellipsoid — and the $R$ in that expression is the radius this essay has been fitting. A radius wrong by 2,200 parts per million makes the reduction wrong by 2,200 parts per million of the height ratio, which for a survey at 300 metres is a sixth of a millimetre in a kilometre and is nothing.

But the same $R$ appears again, and there it is not nothing: the reduction from the reference surface to a grid goes through the projection’s scale factor, and the projection’s scale factor is computed on the ellipsoid the grid was defined on. Change the ellipsoid and the grid coordinates move, by an amount that has nothing to do with the height of the ground. That is why a national grid is inseparable from its national ellipsoid — the two are one artefact — and it is why adopting a global datum meant redefining the grid rather than merely re-labelling it.

The ground is not the grid works through both reductions and the elevation at which they cancel.

The same fit at a different size #

Three points on one trade is enough to see its shape: ten degrees of latitude gives a factor of seven, thirty gives less, and the whole Earth gives nine per cent. Nothing about the method changes between them.

Who found it, and when #

The national ellipsoids belong to the nineteenth century’s great triangulations, and each carries the name of whoever adjusted the arcs.

George Biddell Airy computed his in 1830 from British and continental arcs. Friedrich Bessel published his in 1841 from ten arcs across Europe, Russia, India and Peru — an unusually broad set for its date, which is why it is a better global figure than most of its contemporaries and is still in use in several countries. Alexander Ross Clarke produced figures in 1858, 1866 and 1880 for the Ordnance Survey, the 1866 one becoming the basis of North American mapping for a century.

John Fillmore Hayford’s 1909 figure — adopted as the International Ellipsoid in 1924 — is the one that broke the pattern, and it broke it in an interesting way. Hayford fitted not to arcs but to deflections of the vertical across the United States, with an isostatic correction applied to remove the effect of topography. That is a fit to gravity rather than to geometry, and it is the beginning of the line of work that ends at the ellipsoid is a level surface, where the figure stops being a shape chosen to fit and becomes a consequence of the body’s mass and spin.

The proliferation ended for a practical reason rather than an intellectual one. Once orbits had to be computed, a geocentric frame was compulsory: a satellite does not know which country is beneath it, and its ephemeris is a statement about the centre of mass of the Earth.

Where this goes next #

Fitting a figure to a region is one half of what a datum is. The other half is fitting a transformation between two regions’ figures, which is where the arithmetic stops being kind: where a fit leaves residuals shows what seven parameters cannot reach, and why two authorities can publish different parameters for the same pair of datums and both be right.