Molodensky's shortcut

The exact way to move a coordinate from one datum to another has four steps, and only one of them is the transformation. Convert to Cartesian coordinates on the source ellipsoid; apply the seven parameters; convert back to geographic coordinates on the target ellipsoid; and that last step has no closed form.

There is no algebraic expression for latitude given $(X, Y, Z)$ on an ellipsoid. The standard route iterates — guess a latitude, compute the radius of curvature, get a better latitude, repeat — and converges in half a dozen passes. That is nothing today and it was not nothing in 1958, when the same conversion had to be done for every point in a survey on a machine that measured its memory in kilobytes.

So a direct formula was derived: apply the shift straight to the latitude, the longitude and the height, and never form a Cartesian coordinate at all.

The formulae #

Molodensky’s expressions are a first-order expansion of the exact route in two small quantities: the shift itself, and the difference between the two ellipsoids. Written out, the latitude increment is

$$(M + h)\,\Delta\varphi = -\Delta x \sin\varphi\cos\lambda - \Delta y \sin\varphi\sin\lambda + \Delta z \cos\varphi + T$$

$$T = \Delta a\,\frac{N e^2 \sin\varphi\cos\varphi}{a} + \Delta f\left(M\frac{a}{b} + N\frac{b}{a}\right)\sin\varphi\cos\varphi$$

with the longitude and height increments following the same pattern. The structure is readable even without following the algebra. The first three terms project the translation vector onto the local north direction. The ellipsoid term $T$ is split out on its own line because it is where the two variants of the method differ: it carries one contribution for the change in size and one for the change in shape, and each is weighted by its own radius of curvature, because a change in $a$ and a change in $f$ move the surface normal differently.

Everything is evaluated on the source ellipsoid at the original latitude. Nothing iterates. That was the point.

What was computed, and how #

The measurement is a comparison, and it is only worth anything because the exact route is available to compare against.

For each of five latitudes and three heights, the point is transformed twice: once through Cartesian coordinates with the full iterative inverse, and once through Molodensky. The two results are then differenced on the ground, in metres — not in arcseconds, because the question is whether the shortcut is good enough for a job and a job has a tolerance in metres.

The result for NAD27, whose transformation is a pure translation of about 240 metres:

Seven millimetres across every latitude from 15° to 75° and every height from sea level to three kilometres. That is a first-order expansion behaving exactly as a first-order expansion should: the neglected terms are of order the square of the shift over the radius, which is $(240/6.4\times10^6)^2 \times 6.4\times10^6$, or about nine centimetres — and the measured error is an order of magnitude better than that bound because the leading neglected term partially cancels.

The abridged form is a different formula #

The second row of that table is the finding, and getting it required fixing a mistake in the machinery first.

The first version of this comparison implemented “abridged Molodensky” as the full formulae called with the height set to zero. That is what the name suggests — the abridged form drops the height terms — and under it the abridged version came out 8 per cent worse than the full one. Eight per cent is not a finding. It is a difference too small to justify two names.

The abridged form is not the full formulae at zero height. It makes three simplifications, all of them for hand computation:

• the height is dropped from the two denominators, $M + h$ and $N + h$;
• the two ellipsoid-difference terms — one carrying $\Delta a$ with its own radius, one carrying $\Delta f$ with a different combination — are replaced by the single combined coefficient $(a\,\Delta f + f\,\Delta a)\sin 2\varphi$;
• the height increment loses its $\Delta a$ weighting in the same way.

The second of those is the one that costs. Two coefficients that differ with latitude have been replaced by one that does not, so the substitution is exact at whatever latitude the two happen to cross and wrong on either side. That crossing is near 45°, which is the dip in the hero figure, and it is why the abridged form has a reputation for being adequate: a great deal of the world it was applied to is at middle latitudes. The same accident runs through this subject — what a standard parallel buys is a deliberate version of it, where a projection is made exact on two lines so that the error between them has both signs and a smaller worst case.

Comparing the two figures is why this generator takes the datum as its parameter. A method’s error is a property of the method and of what it is applied to, and a figure drawn for one transformation would be making a claim about a formula while showing a single case.

The figure this generator refuses to draw #

Both figures above are three-parameter datums, and that is not a presentational choice. Molodensky’s formulae have no rotation term and no scale term at all. They carry three translations and two ellipsoid differences, and there is nowhere in them for the other four parameters to go.

So a datum whose published transformation needs seven — OSGB36, whose rotations and scale contribute 150 metres between them — cannot be expressed by these formulae. The formulae do not object. They accept the three translations, ignore the rest, and return a coordinate of exactly the right shape.

The size of that mistake was measured when this essay’s figure was first drawn for OSGB36 and the generator’s own assertion failed. The assertion says the full form must beat the abridged one; it reported 27.5 metres against 27.4 — indistinguishable, because both were swamped by the four parameters neither of them carries.

Three consequences, and the third is why the refusal is worth having rather than a footnote:

• the error from misapplying the method is 15 to 27 metres, against the method’s own approximation error of 7 millimetres — a factor of two thousand;
• the two variants become indistinguishable, differing by under 5 per cent of a number that is entirely wrong, so the usual way of judging the method reports nothing;
• and the failure is silent without the check, because a formula that takes $\Delta x$, $\Delta y$, $\Delta z$, $\Delta a$ and $\Delta f$ has no way to notice that its caller had seven numbers and gave it three.

The generator now refuses a seven-parameter datum by name, and the library carries the misuse as a deliberate measurement rather than as a caveat.

Why the inverse has no closed form #

The step Molodensky exists to avoid deserves its own look, because the reason it is hard is the same reason this whole subject exists.

Going to Cartesian coordinates is direct: given a latitude, the prime vertical radius $N$ follows, and the three components follow from that. Going back is not, because the latitude appears inside $N$ and $N$ appears in the equation for the latitude. The geodetic latitude is defined by the direction of the normal to the surface, and the normal at a point does not pass through the centre — which is the whole difference between an ellipsoid and a sphere, and the reason geodetic and geocentric latitude differ by up to eleven arcminutes.

On a sphere the inverse is trivial: the latitude is the angle at the centre and one arctangent gives it. On an ellipsoid the same arctangent gives the geocentric latitude, which is a different angle by up to 11.5 arcminutes — about 21 kilometres on the ground. So the closed form that exists is the closed form for a quantity nobody wants, and the quantity everybody wants requires iteration or a series.

There are closed-form solutions for the ellipsoidal case, by the way — Vermeille’s and Ferrari’s among them — obtained by reducing the problem to a quartic. They arrived decades after Molodensky and they are exact. That they exist is a useful corrective to the idea that the shortcut was necessary rather than expedient.

What the shortcut is worth against what it is applied to #

Setting the two numbers side by side is the fair way to judge an approximation, and it flatters this one. The shift is 190 metres; the approximation’s error is 7 millimetres. On any reasonable reading the formula is exact.

The trouble with that reading is that nobody consumes the shift. What gets consumed is the corrected coordinate, and its error is not the error in the correction — it is the error in the correction plus everything the correction could not model. Ranking the three:

• the shift itself: 190 metres, modelled;
• the residual of the seven-parameter fit: one to two metres, not modelled;
• Molodensky’s approximation: 7 millimetres.

The middle row is two hundred times the bottom one. Which means the honest way to describe the full Molodensky formulae is not “accurate enough” but “irrelevant to the error budget” — and the abridged form’s 380 millimetres is also below the residual, so on this argument even the bad shortcut is defensible. It is defensible right up until the transformation is exact, which is the case when two datums are related by definition rather than by a fit, and then the 380 millimetres is the whole error.

Where the model stops #

This measures the shortcut, not the parameters. Both routes use the same published seven parameters, so both inherit whatever is wrong with them. What is being compared is two ways of applying the same transformation, and the residual left by the transformation itself — metres, and structured, as where a fit leaves residuals measures — is two orders of magnitude larger than the seven millimetres here.

That ordering is worth stating plainly, because it changes what the seven millimetres means. Molodensky’s approximation is not the limiting error in any real conversion. It is a hundred times smaller than the residual of the fit it is applying. Choosing the full form over the abridged one is still worth doing — 380 millimetres is not negligible against a boundary survey — but choosing the exact Cartesian route over the full Molodensky is choosing between seven millimetres and zero, underneath a metre and a half of unmodelled network strain.

There is a trap in that figure worth naming, because it is a larger error than anything else in this essay. The formulae take ellipsoidal height. The height printed on a map, quoted by a surveyor or read off a barometer is orthometric, and the two differ by the geoid separation — tens of metres, and up to a hundred. Supplying the wrong one is a height error of the same size, and the resulting horizontal error is that separation divided by the Earth’s radius times the shift, which is small. So the mistake is nearly harmless horizontally and directly harmful in the vertical, which means it survives every check that looks at positions on a map. Height above what? is the essay about the two heights.

The comparison is over one set of latitudes and three heights. The formulae are smooth in both, so the sampling is adequate, but the worst case is over the sampled range rather than proven over all of it.

Three-parameter transformations only. The Molodensky formulae carry the three translations and the two ellipsoid differences. They have no rotation or scale terms at all. So for a datum like OSGB36, whose rotations and scale contribute 150 metres between them, the formulae have to be preceded by applying those separately or the answer is wrong by far more than any approximation error. The abridged form’s 380 millimetres is a rounding next to that, and it is exactly the kind of error that gets made — because a formula that takes $\Delta x$, $\Delta y$, $\Delta z$, $\Delta a$ and $\Delta f$ silently accepts a datum whose transformation needs seven numbers and returns something that looks like an answer. That is the same silence what a coordinate refers to is about: none of the failure modes in this field produces an error message.

What it costs to run the exact route instead #

The case for the shortcut was arithmetic cost, so it is worth asking what that cost actually is now.

The iterative inverse converges quadratically once it is near the answer, and a good starting value — from the parametric latitude — puts it near the answer immediately. Six passes reach the limit of double precision. Each pass is a square root, a division and a couple of trigonometric calls. On a modern processor the whole conversion is a few hundred nanoseconds, and converting every point in a national dataset of ten million marks takes a few seconds.

Against that, Molodensky saves perhaps a factor of five in a step that is not the bottleneck of anything. The argument for it in 2026 is not speed. It is that the formulae are printed in a standard, implemented in a library, and produce answers close enough that nobody notices — which is a reasonable description of most of the code in this field.

There is one place where the direct form is still genuinely the right choice: when only the shift is wanted rather than the transformed coordinate. Molodensky gives $\Delta\varphi$ and $\Delta\lambda$ directly, so a shift table can be built from it without the round trip, and a shift table is what the next essay is about.

The generalisation #

There are two portable lessons here and they point in opposite directions, which is what makes the pair worth having.

A first-order expansion is entitled to a second-order error, and that is a claim that can be checked. The neglected terms in Molodensky are $O((\Delta/R)^2 R)$, which is nine centimetres; the measured error is seven millimetres. An expansion whose measured error is much larger than its own bound has an implementation mistake in it, and an expansion whose error is much smaller usually has a cancellation worth understanding. Either way, the bound is a prediction and the measurement tests it. That is the same discipline the transverse Mercator series is held to, and the same one behind the Chebyshev criterion’s refusal to accept a measured spread that came out better than a proved bound, where the series’ truncation error is computed rather than assumed and an independent quadrature caught a coefficient applied to the wrong term.

A simplification with a name is a different formula and needs its own measurement. “Abridged” sounds like the same thing with fewer digits. It is a substitution of one coefficient for two, and its error is fifty times larger and has a completely different shape — exact at one latitude and growing away from it. The instinct that treats a variant as a rounding of its parent is the same instinct that made this essay’s first implementation wrong, and it was caught by the same thing that catches everything here: implementing both and measuring, rather than reasoning about which should be better.

The general form is worth stating as a rule. When two methods have a common ancestor, implement both from their own definitions. Deriving one by disabling a feature of the other tests the disabling, not the method.

The shortcut across a spread of ground #

What the formulae are approximating varies across a country by tens of metres rather than being a level to be reproduced once, and Molodensky reproduces the variation as well as the level. That is the part of the claim a single-point comparison cannot make, and it is the reason the method survived the arrival of better data rather than being replaced by a table of local offsets.

Comparing the sea-level and the three-kilometre figures isolates what the abridged form actually gave up. Its combined ellipsoid coefficient costs the same at any height; the height terms cost nothing at sea level and grow linearly above it, which is why the shortcut has a reputation that depends entirely on where it was used.

Who found it, and when #

Mikhail Sergeevich Molodensky worked at the Central Research Institute of Geodesy in Moscow, and is better known for a deeper contribution than this one: the theory of determining the physical surface of the Earth from surface measurements alone, without any assumption about the density of the rock beneath — which is what makes a levelled height computable without a hypothesis about the interior.

The transformation formulae are a minor piece of that work and became the widely used piece, because they arrived exactly when large numbers of coordinates first had to be converted between datums and computers were expensive. They were adopted into the United States’ military mapping standards, and from there into more or less every geographic information system, which is why a formula optimised for the constraints of 1958 is still the default in software running on machines a hundred million times faster.

The persistence is not laziness so much as inertia in a standard. And the abridged form’s persistence is worse: it is the version printed in the mapping standard’s appendix, so it is the version implemented, and its extra 373 millimetres are inside a great deal of converted data with nothing to indicate them.

Where this goes next #

Both routes in this essay assume the relationship between two datums is a transformation. It is not, quite — it is a transformation plus a field, and the field is why national agencies ship tables. When a formula is not enough works out what those tables cost, and finds the answer is set by the roughness of the difference rather than by its size.