The height a coordinate does not carry

Ten rungs of this anchor treat a datum transformation as a map from one pair of angles to another: a latitude and a longitude go in, a different latitude and longitude come out, and the difference is the shift.

That is not what a Helmert transformation is. It is a rigid motion of three-dimensional space, and it acts on a Cartesian coordinate — so before it can be applied, a latitude and a longitude have to be turned into a point, and turning an angle pair into a point requires a height.

A map sheet does not have one. A boundary description does not have one. Most of the world’s spatial databases hold a latitude and a longitude and nothing else. And the answer depends on the number that is missing.

What the convention is, and what it hides #

Every practitioner knows the convention: for a two-dimensional coordinate, put in a height of zero and take out a latitude and a longitude. It is universal, it is what every library does by default, and it works — in the sense that it returns an answer that is right if the point really is on the ellipsoid.

The question nobody asks is what it costs when the point is not.

The relation is a straight line through the origin, to a part in seven hundred from a doorstep to the summit of Everest. That is a convenience rather than a subtlety: the sensitivity at a place is one number, in millimetres per kilometre, and it would fit in a parameter file beside the seven that are already there.

There is one more thing the linearity buys, and it is the reason the number is worth publishing rather than merely knowing. Because the relation is linear and passes through the origin, an uncertainty in the height converts to an uncertainty in the horizontal position by the same constant — so a practitioner who knows their elevation to within two hundred metres knows their transformed position is uncertain by six millimetres from that source alone, and can compare it against everything else in their error budget. A non-linear relation would need a table; this needs a multiplication.

Four hundred and thirty-two millimetres is not a rounding error. It is four times a first-order survey’s positional tolerance, it is larger than the width of the boundary line on any plan it would be drawn on, and it arises from substituting a number the data does not contain.

Who is exposed, and by how much #

The rates are abstract until they are put beside the thing they threaten, so it is worth naming the cases.

A coastal or low-lying survey is safe. Below about a hundred metres of elevation the largest rate here gives 12 millimetres, which is under a first-order positional tolerance and far under what any historical mapping of a datum this old was captured to.

A plateau or a mountain country is not. Denver at 1,609 metres, Zermatt at 1,608, Quito at 2,850 and Lhasa at 3,656 are ordinary inhabited places, and every one of them exceeds a decimetre under at least one of these four transformations. The whole of the Bolivian altiplano, the Ethiopian highlands, the Tibetan plateau and the American intermountain west are above 1,500 metres; so is most of Switzerland’s inhabited area above the lake basins.

And the exposure is exactly inverted relative to attention. A transformation is checked most carefully where the surveying is densest, which is where the land is flat and the effect is smallest; it is applied without checking in the mountains, where the effect is largest, because that is where the control points are sparse and nobody has a residual to notice it in. The error is smallest where somebody would find it and largest where nobody is looking, which is the shape of every defect this collection is written about.

Four mechanisms, measured one at a time #

The size varies by a factor of sixteen across the twenty cases, so “the rate” is not a property of a transformation. Turning each mechanism on by itself says why.

Dilution. The transformation’s Cartesian offset is converted back into an angle by dividing by a radius, and the radius carries the height — so a fixed offset subtends a smaller angle at a higher point, and the ground displacement shrinks by $h/M$. The rate is the horizontal part of the translation over the Earth’s radius, and it is present between two copies of one ellipsoid: a pure translation of 917 metres between two WGS84s gives 47 millimetres per kilometre at London. That is the term nobody expects, and finding it is what turned this rung’s expected control into a measurement.

Shape. Two ellipsoids of different flattening have normals that differ in direction by about $\Delta f \sin 2\varphi$, so moving along one is moving sideways relative to the other. It is the largest term for NAD27, whose Clarke 1866 ellipsoid differs from WGS84 in flattening by more than any other in the library.

Rotation. Nothing, to two decimal places: 0.02 per cent of the total on OSGB36 and on DHDN, the two transformations with rotations large enough to see anywhere else. The reason is exact rather than numerical — rotating the point and rotating the displacement are the same rotation, so a rotation moves everything together and produces no height dependence at all beyond a term in the flattening.

Scale. 0.3 per cent at most. A scale change moves a point radially, and on an oblate body a radial move is not quite along the normal — so the term exists and is a hundred times smaller than the two that matter. It is the only one of the four that can be dropped.

The ordering is worth stating plainly because it inverts what the parameter list suggests. A reader looking at seven published numbers would expect the rotations and the scale to be where the subtlety lives, since those are the parameters that are hard to interpret and that have two sign conventions. They contribute a third of a per cent between them. The height sensitivity is carried entirely by the translation, which is the part of the transformation everybody understands, and by the two ellipsoids, which are not in the parameter list at all.

The four are vectors and they partly cancel. NAD27 at Denver has a dilution term at 488 per cent of the answer and a shape term at 480 per cent pointing the other way, leaving 7.7 millimetres per kilometre — the smallest rate in the table, produced by the two largest contributions in it.

It is a field, and it has a direction #

Across the sphere one transformation’s rate varies by a factor of twenty-five, so it cannot be quoted per datum any more than per transformation. And the displacement is not merely a magnitude.

That rules out the obvious repair. A systematic error in one direction could be folded into the translation parameters and forgotten; this one points north-west at Lhasa, south-west at Denver and south-east at London under the same transformation, so no adjustment of the seven parameters can absorb it. It is not a parameter error. It is a missing input.

What it does to a round trip #

There is a second-order consequence that is worse than the first-order one, and it is the reason this matters for archives rather than only for surveys.

Transform a two-dimensional coordinate with an assumed height of zero, and transform it back the same way. The round trip is exact: the same wrong height goes in both times and the errors cancel, so a coordinate that goes out and comes back is unchanged. That is reassuring and it is the check most software runs.

Now transform it out with a height of zero, store the result, and transform it back later from a system that supplies a real height — a digital elevation model, say, which is a routine enrichment. The two halves no longer use the same height, the errors no longer cancel, and the coordinate moves by the full amount. The failure appears when the data is improved.

That is the version of this defect that is hardest to catch, because every test of the transformation in isolation passes and the round trip only breaks once two parts of a pipeline disagree about a number that neither of them was told to record. A chain of transformations does not close measures the same shape for the parameters themselves; this is the same statement about an input the parameters do not include.

What was computed, and how #

Each case is the transformation run twice at the same latitude and longitude with two different heights, converted back to geodetic coordinates on WGS84, and differenced on the ground in metres. Nothing is linearised and no formula is fitted: the numbers are the difference between two full transformations.

The decomposition is measured rather than derived. Each mechanism is isolated by running the same comparison with the other three switched off — a translation-only transformation between two copies of WGS84 for the dilution, a zero-parameter transformation between the two real ellipsoids for the shape, and so on. An algebraic decomposition was tried first and was wrong by nineteen per cent, because the three terms interact through the geodetic conversion in a way the leading-order algebra does not capture. The four isolated runs sum as vectors to the measured total to within a fiftieth of a per cent, which is the check that the four are the whole of it.

The assertions require six things separately: that the displacement be linear in the height to a part in two hundred over three decades; that it be non-zero for a translation-only transformation with no rotations and no scale; that the four isolated mechanisms sum to the total within two per cent everywhere; that the shape term vanish exactly between identical ellipsoids and the scale term be negligible; that no single term dominate everywhere, which is what makes the decomposition necessary; and that at least one real place at its real height move by more than a centimetre.

What this collection’s own figures assume #

The audit belongs here for the same reason it belonged in four radii of the Earth: a rule about somebody else’s practice is worth less than the same rule applied to the machinery in front of the reader.

Every datum figure on this site transforms points at a height of zero. Datum shifts dwarf projection errors reports shifts of hundreds of metres; where a fit leaves residuals reports residuals of metres; the seven parameters reports the contribution of each parameter in metres. All of those are at $h = 0$, and the correction at a real elevation is a few decimetres.

Two of those three are unaffected in substance — a shift of 700 metres is not changed by a decimetre, and the shares the seven parameters contribute are ratios. The third is not: a residual of a metre and a correction of four decimetres are the same order, so a fit residual measured at zero height is measuring the fit and the height assumption together wherever the control points are high.

That is the honest state and it is recorded rather than repaired, because repairing it means giving every control point a height, which those essays’ point sets do not have. It is in the site’s own shortfall list.

Where the model stops #

The height in question is the ellipsoidal one. Height above what? establishes that what a database usually holds, if it holds anything, is an orthometric height — and the two differ by the geoid separation, which is tens of metres. So a practitioner who does supply a height may be supplying the wrong kind of one, and a fifty-metre geoid separation is worth another few millimetres of horizontal error on top of everything measured here.

A grid-based transformation behaves differently. When a formula is not enough covers the case where the shift is published as an interpolated grid rather than as seven parameters, and a two-dimensional shift grid is defined on latitude and longitude alone — so it has no height dependence at all. That is a genuine advantage of the grid form which nobody lists among its advantages, and it is the reason the effect measured here is invisible in the countries that use one.

And the four datums here are the four this collection carries. Their translations run from 200 to 900 metres and their flattening differences from zero to 3.7 × 10⁻⁵, which spans the usual range; a transformation between two modern realisations, whose translations are centimetres, would have a rate a thousand times smaller and entirely negligible. The effect is a property of old datums, which is to say of exactly the transformations that historical mapping needs.

The generalisation #

The rule is that a function of three variables applied to two of them has silently been given a value for the third, and the value is usually zero because zero is what an uninitialised variable is.

Nobody decides to assume sea level. The transformation’s signature takes a height, the calling code has none, and a zero goes in — which is a decision made by a function signature and a default argument rather than by anybody. That is the same shape as the sample drawn on the page, where the domain of an average arrives with the data structure, and as the pooled score abandons a region, where a weighting arrives with the length of an array.

The cheap repair is to publish the rate. It is one number per transformation per place, it is exactly linear so it needs no table, and it turns an invisible assumption into a stated uncertainty: at this place, a metre of height error is 0.03 millimetres of horizontal error, so a hundred metres of unknown elevation is three millimetres and can be ignored, and three kilometres is a decimetre and cannot.

The habit generalises to any conversion between coordinate systems of different dimension. Ask what the extra coordinate was set to, and what the answer’s sensitivity to it is. If nobody can say, the conversion has an unquantified error whose size is a property of the place rather than of the software.

Who found it, and when #

This is not a discovery and it is not disputed. Every geodesy text that presents the Helmert transformation presents it in three dimensions, and the good ones say explicitly that a two-dimensional application needs an assumed height. Software documentation frequently says so too.

What is missing is the number. The warning is qualitative — the height affects the result — and a practitioner reading it has no way to decide whether it matters for their job, because the size is nowhere. A warning without a magnitude is indistinguishable from a warning about a rounding error, and gets treated as one.

The place the profession does take it seriously is in the countries that abandoned parameters for grids. A national shift grid is two-dimensional by construction, defined on latitude and longitude alone, and switching to one removes this problem completely along with the fit residuals it was actually adopted to remove. That is a real advantage arrived at for a different reason, and it is not in the lists of why grids are better.

So the practical statement is short. A seven-parameter transformation of a two-dimensional coordinate is not defined until a height is supplied, the sensitivity is between 8 and 120 millimetres per kilometre depending on where and which, and for anywhere above about two hundred metres it exceeds the tolerance the coordinate was probably surveyed to.

Where the ladder goes next #

Eleven rungs have measured what a datum transformation does and what it needs. Every one of them has taken the parameters as given — published numbers, adopted by a national agency, applied as stated. What none has asked is what happens when a country publishes a second set for the same pair of datums, which several have, and which of the two a coordinate was transformed with is not recorded anywhere in the coordinate.