The plumb line is not the normal

Before satellites, the way to find out where a ship or a survey station was was to look up. Time a star’s transit, measure its altitude, and out comes a latitude and a longitude — determined by the direction of the plumb line against the sky, because every instrument that measures an angle from the vertical is levelled by a bubble and a bubble finds gravity.

That is astronomic latitude. It is not the latitude a coordinate means.

The claim #

Geodetic latitude is the angle between the equatorial plane and the normal to the ellipsoid — a mathematical direction, computable from four constants. Astronomic latitude is the angle between the equatorial plane and the plumb line — a physical direction, determined by where the rock is.

The angle between the two is the deflection of the vertical, and it is not small in the units that matter: a few arcseconds is ordinary, tens of arcseconds happens in mountains, and every arcsecond is 31 metres of ground.

The deflection is a slope, exactly #

The relation between the deflection and the geoid is the cleanest thing in this subject, and it needs no approximation.

The plumb line is perpendicular to the geoid, by definition — that is what an equipotential surface means. The ellipsoid normal is perpendicular to the ellipsoid. So the angle between the two directions is the angle between the two surfaces, which is the slope of the geoid relative to the ellipsoid:

$$\xi = -\frac{\partial N}{\partial s_{\text{north}}}, \qquad \eta = -\frac{\partial N}{\partial s_{\text{east}}}$$

with $\xi$ and $\eta$ the north and east components. A dimensionless slope, read as an angle.

Which makes the conversion arithmetic. One arcsecond is $4.848 \times 10^{-6}$ radians, so:

• one arcsecond of deflection is the geoid rising 4.85 millimetres in a kilometre;
• ten arcseconds is 48.5 millimetres per kilometre;
• and a deflection of one arcsecond displaces an astronomically determined position by 31 metres on the ground.

That last number is the one to hold. An arcsecond is thirty metres, and a deflection of ten arcseconds — which is unremarkable in hill country — puts a star-sighted position 309 metres from where the coordinate says.

What was computed, and how #

Everything in this essay is a linear relation and is exact, which makes the computation a matter of stating it in units somebody can act on rather than deriving anything.

The ground displacement is the deflection times the radius:

$$\Delta s = \xi \cdot R$$

with $R$ the appropriate radius of curvature. The figures take the deflection as their parameter for a good reason: the relation has no scale in it. There is no characteristic deflection, no natural size, nothing that distinguishes one arcsecond from thirty except which piece of ground is being stood on. A figure drawn at a single value would be showing a special case of something that has none.

What is not computed here is the deflection at any particular place, and that is deliberate. The deflection is the gradient of the geoid, and the geoid is a data product — the decision recorded in height above what?. So the relation is computed and the value is the reader’s own.

Why this is what made a datum local #

The deflection is the mechanism behind the third of the four commitments in what a coordinate refers to, and following it explains a great deal about why national datums were placed where they were.

A classical datum was established by choosing one station, observing its position astronomically, and declaring that observation to be the geodetic position. That is a decision, not a measurement: it sets the deflection to zero at that point by fiat, and everything else in the network is computed outward from it by triangulation.

Two consequences follow immediately.

The ellipsoid ends up displaced. Fixing the geodetic latitude to the astronomic one at the origin ties the ellipsoid’s orientation to the local plumb line, and the local plumb line is tilted with respect to the geocentric normal by whatever the geoid is doing there. The result is an ellipsoid whose centre is hundreds of metres from the Earth’s — the translations in the seven parameters.

The choice of origin was a real engineering decision. The best origin is one where the deflection is small and typical of the region — away from mountains, away from coastlines, in the middle of the ground being mapped. Getting it wrong tilts the whole national frame.

The correction is available: observe astronomic positions at many stations, compare with the geodetic positions carried by triangulation, and the differences are the deflections. That comparison is an astro-geodetic survey, and it is how the geoid was mapped before gravimetry could do it — Hayford’s 1909 ellipsoid, mentioned in a datum is fitted to a region, was fitted to deflections rather than to arcs, which is a fit to gravity rather than to geometry.

Two more angles called latitude #

This subject already has six angles competing for the word, and the astronomic one is a seventh — different in kind from the rest, because it is the only one that is not a function of the geodetic latitude.

That is the distinction worth carrying. The parametric, authalic, conformal, rectifying and geocentric latitudes are all functions — feed in a geodetic latitude and out comes a number, exactly, with no reference to the planet. Geodetic against geocentric latitude works through the largest of those gaps, which reaches 11.5 arcminutes.

The astronomic latitude is a measurement, and it differs from the geodetic one by an amount that no formula can supply. Eleven arcminutes of geocentric–geodetic difference is larger than ten arcseconds of deflection by a factor of seventy, and the geocentric gap is exactly known while the deflection is not — which is the whole difference between a construction and an observation.

The other consequence: an azimuth #

There is a second effect, and it is the one that used to bite hardest in practice.

An azimuth observed astronomically — from a star, or from the sun — is also referred to the plumb line, so it too differs from the geodetic azimuth. The relation is Laplace’s equation:

$$A - \alpha = \eta \tan\varphi$$

The east component of the deflection times the tangent of the latitude. That factor is what makes it dangerous: at 60° latitude, $\tan\varphi$ is 1.73, so ten arcseconds of east deflection is seventeen arcseconds of azimuth error. Carried through a triangulation chain of several hundred kilometres, an azimuth error of seventeen arcseconds is tens of metres of lateral displacement.

The remedy was the Laplace station: a point where both the astronomic azimuth and the astronomic longitude are observed, so that the equation can be applied and the chain’s orientation reset. National triangulations are studded with them at intervals of a few hundred kilometres, and their spacing was set by exactly this arithmetic.

That figure carries the distinction the whole essay turns on. The separation $N$ and the deflection $\xi$ are different quantities: one is the height of the geoid, the other is its slope. A region can have a hundred metres of separation and no deflection, or none of the first and a great deal of the second. The separation corrupts heights; the slope corrupts directions.

What it costs today #

Astronomic position determination is obsolete for navigation and is not obsolete for geodesy, and the deflection is the reason.

A satellite receiver gives geodetic coordinates directly, so nothing in ordinary positioning meets the deflection at all. What still meets it is anything that is levelled, because levelling is still done with a bubble, and a bubble still finds gravity:

• a total station measures vertical angles from the plumb line, so trigonometric heighting over long sights inherits the deflection;
• an inertial navigation system integrates accelerations in a frame it establishes by sensing gravity, so its idea of vertical is the plumb line and its position drifts by the integrated deflection;
• a precise levelling run is affected as a levelled height is not a distance describes, through a different mechanism — the spacing of the level surfaces rather than their tilt.

The inertial case is the one that has grown rather than shrunk. A navigation system that has lost its satellite signal is running on gravity, and its error budget contains a term that is the integral of the deflection along the route — which is why high-grade inertial systems carry a geoid model.

Where the model stops #

No deflection is computed here. The relation between slope and angle is exact and is what this essay computes; the actual slope at a place requires a geoid model, which this site treats as a citation. Anybody wanting a number for a particular point needs a national or global model and should note which one.

The deflection has two components and this essay mostly treats one. The north component $\xi$ affects latitude, the east component $\eta$ affects longitude and, through Laplace’s equation, azimuth. They are independent, and a place can have a large one and a small other.

Deflections are not small near topography. A few arcseconds is a lowland figure. Values of thirty to sixty arcseconds occur near mountain fronts, and the extremes are larger. So the 309 metres in the hero figure is a moderate case rather than a worst one.

Nothing here is about refraction. An astronomical observation also has to contend with the atmosphere bending the light, which is a separate correction of comparable size and completely different origin. The two were historically confused, and separating them is part of why deflections took so long to be believed.

The generalisation #

The portable statement is about instruments defining their own reference, and it is uncomfortable once seen.

Every instrument that is levelled — a theodolite, a level, a sextant with an artificial horizon, a total station — is referred to the local gravity vector, because that is what levelling means. So all of classical surveying is conducted in a frame defined by the rock nearby, and the mathematical frame everybody writes their answers in is a different one. The two agree where the geoid is parallel to the ellipsoid and nowhere else.

That is not a defect in the instruments. It is the observation that a measurement’s reference is whatever the measuring device physically responds to, and the reference stated in the write-up is often a different thing that has been assumed equivalent. The equivalence is usually good and is never exact, and the size of the gap is a measurement in its own right.

Two more instances of the same shape, in this collection and outside it. Grid north is not north is the same statement one layer up: a grid bearing, a true bearing and a magnetic bearing are three different angles at the same point, and only the middle one is a property of the Earth. And a thermometer measures its own temperature rather than the air’s, which is why the shelter it sits in is part of the specification.

The general remedy is also the same in all three cases: observe the difference at enough places to map it, then correct. Laplace stations, calibration baselines and instrument shelters are all the same idea.

What a deflection does to a projected coordinate #

Everything so far is about the datum. It is worth chasing one step further, into the part of this collection that is about maps, because the answer is reassuring in a way that is easy to misread.

A projection takes a geodetic latitude and longitude and produces an easting and a northing. It has no opinion about the plumb line, so a deflection does not distort a map: it displaces the observation that produced the coordinate, once, at the time of observation. A coordinate obtained from triangulation carried from a properly oriented network is unaffected.

So the deflection is a nineteenth- and twentieth-century problem in one sense — it corrupted the way positions used to be found — and a current problem in another, because levelled instruments are still levelled. What it is not, at any point, is a mapping problem. That distinction is the same one datum shifts dwarf projection errors draws: the errors people argue about live in the projection, and the large ones live upstream of it.

A small deflection, and what it still costs #

Three figures at three deflections, and the relation between them is exactly linear, which is the point of drawing more than one. There is no threshold below which a deflection stops mattering; there is only the tolerance of the work set against 31 metres per arcsecond.

It is worth separating this from the departure underneath it. The ellipsoid’s own normal is already not the direction to the centre — that is what geodetic latitude means — and the deflection of the vertical is a second departure sitting on top of the first. The two are routinely confused, because both are angles of a few arcminutes between things a reader would call “up”.

Who found it, and when #

The deflection was found as an anomaly and explained as a mass.

In the 1730s Pierre Bouguer, measuring an arc in Peru, found that a plumb line near Chimborazo was pulled towards the mountain — by much less than the mountain’s bulk suggested, which is the first evidence for isostasy and a story of its own. In the 1730s and 1740s the same effect turned up in the French arc measurements as inconsistencies nobody could attribute to the instruments.

The decisive case was the Great Trigonometrical Survey of India. In the 1850s the difference between the astronomically determined and triangulated positions of Kalianpur and Kaliana came out at about five arcseconds — far too large for the observations — and the Himalayas were the obvious suspect. John Henry Pratt computed the expected deflection from the mountains’ mass and got a number three times too large, which was as important as the discrepancy itself: the mountains had to be compensated by a deficit of mass beneath them. Pratt and George Biddell Airy proposed two different mechanisms in 1855, and the argument between their models is still live in geophysics.

So the deflection of the vertical is one of those quantities whose error was the discovery. Nobody set out to measure the geoid’s slope; they set out to survey India, found their astronomy and their triangulation disagreeing, and the disagreement turned out to be about what is under the Himalayas.

Why the discovery had to come from a survey #

That the deflection was found as an error rather than as a measurement is worth a moment, because the shape of it explains why it could not have been found any other way.

The quantity has no instrument of its own. Nothing directly reads the angle between a plumb line and an ellipsoid normal, because the ellipsoid normal is not a physical direction — it is a property of a mathematical surface that has to be established by triangulation from somewhere else. So the deflection is only ever a difference between two determinations of the same station’s position, one astronomical and one geodetic.

Which means it requires both to be good. A survey whose triangulation is worse than five arcseconds cannot see a five-arcsecond deflection; neither can one whose astronomy is. The Great Trigonometrical Survey found it because it was, by the standards of the 1850s, extravagantly careful in both, and because it ran for hundreds of miles towards the largest mass anomaly on the planet.

And the discovery required the discrepancy to be unexplainable. A five-second disagreement in a sloppier survey would have been absorbed into the error budget and forgotten. It became a finding because the surveyors could say, with evidence, that neither of their two methods was capable of being that wrong — so the disagreement had to be about the Earth.

That is the pattern worth carrying. A quantity with no direct instrument is found in the residuals of something else, and it is found only when the something else is measured well enough that its residuals mean something. The deflection of the vertical, the flattening, the non-hydrostatic bulge and the plate velocities were all discovered this way, and each of them was somebody’s error first.

Where this goes next #

The vertical’s effects on a coordinate have now been taken separately: the separation, the convergence of level surfaces, and the slope. What remains is where a height meets a horizontal measurement on an actual job, which is the reduction from the ground to the grid — the ground is not the grid, where two corrections with opposite signs cancel at exactly one elevation.