The ground is not the grid

Two survey marks are ten kilometres apart. An instrument on the ground between them measures the distance to a millimetre. The grid coordinates of the two marks, differenced, give a distance six metres different.

Both numbers are right. They are distances between different pairs of points on different surfaces, and getting from one to the other takes two corrections that pull opposite ways.

The two reductions #

The elevation factor brings the measurement from the ground down to the ellipsoid. Two marks on a mountain are further apart than their projections onto the reference surface, in the ratio of the two radii:

$$k_e = \frac{R}{R + h}$$

with $R$ the radius of curvature and $h$ the ellipsoidal height. It is always less than one, and it is 157 parts per million per kilometre of elevation — a metre in every kilometre measured, at a kilometre up.

The grid factor takes the distance from the ellipsoid onto the map. Every projection distorts distance somewhere; a grid’s scale factor is the local value of that distortion, and it can be either side of one. UTM’s is 0.9996 on its central meridian, rising past one at 1.61° out and reaching 1.00098 at the zone edge — the arithmetic in UTM and the zone system.

Their product is the combined factor, and it is the only number a surveyor can use, because it is the only one that connects a measurement to a coordinate.

The cancellation #

Since the two factors have opposite signs whenever the grid factor exceeds one, there is an elevation at which they cancel exactly:

$$\frac{R}{R+h}\,k_g = 1 \quad\Longrightarrow\quad h = R\,(k_g - 1)$$

For $k_g = 1.0004$ at latitude 45°, that is 2,551 metres. At that elevation a ground distance and a grid distance are the same number, and the combined reduction can be skipped.

This is not a curiosity. It is a design principle, and several state and provincial grids are built on it. A low-distortion projection is a grid whose scale factor is chosen so that the combined factor is one at the mean elevation of the ground it covers — Wisconsin’s county coordinate systems, Colorado’s, Oregon’s and Minnesota’s are all designed this way. The projection is deliberately made worse in the classical sense, so that the number a surveyor reads off the ground is the number that goes into the coordinate.

The trade is explicit and worth stating: such a grid covers a small area, because the elevation it is tuned to is only the mean elevation over a small area. Fifty low-distortion zones for a state, against one UTM zone for a sixth of a continent. The narrower the purpose, the better the fit — the same trade as a datum is fitted to a region, one layer up the stack.

What was computed, and how #

Three things, and the third one is a refusal.

The radius. $R = \sqrt{MN}$, the Gaussian mean of the meridian and prime-vertical radii, computed at the stated latitude from the ellipsoid’s constants. It varies from 6,357 kilometres at the equator to 6,400 at the pole, so the cancellation height varies with latitude by about half a per cent — small, and computed rather than assumed constant.

The combined factor across height, at a stated grid factor, in parts per million, which is the unit a specification is written in.

The refusal. If the grid factor is below one, there is no cancellation at all: both reductions shrink the measurement, so the combined factor only moves further from one as the ground rises. The machinery returns no cancellation height in that case rather than a negative one, and the assertion requires it — because a formula that cheerfully returns $h = -2551$ metres for a grid factor of 0.9996 would be describing a survey below the ellipsoid.

Comparing the two figures is why the generator takes the grid factor as its parameter. The existence of the cancellation is a property of the grid, not of the method, and a figure that drew one case would be presenting a special case as a general result.

What the grid factor actually is #

The grid factor is not a constant of the grid. It is the projection’s own scale factor evaluated at the point, and on a transverse Mercator it varies across the zone quadratically.

So a survey in the middle of a zone and a survey at its edge have grid factors differing by 1,400 parts per million, and a job that crosses a zone boundary meets a discontinuity. That is the same trade what a standard parallel buys sets out, applied at continental scale: convert a systematic error into a smaller error of both signs.

The trade is repeated in sixty zones, each a separate projection about its own central meridian, and a coordinate carries no record of which one it belongs to — so a combined factor computed in the wrong zone is wrong by up to fourteen hundred parts per million with nothing at all to indicate it.

Which height, and why it matters here #

The $h$ in the elevation factor is ellipsoidal height, and almost nobody has one.

The height a surveyor holds is orthometric — carried from a national levelling network, or read from a map — and the two differ by the geoid separation, which is tens of metres. Using the wrong one introduces an error of $N/R$, which is 45 metres over 6.38 million, or 7 parts per million.

Seven parts per million is seven centimetres on a ten-kilometre baseline. That is under most specifications and over some, and the important part is that it is systematic: it does not average out over a job, it has the same sign everywhere the geoid is high, and it is invisible in any internal check because every measurement is wrong the same way.

This is the one place in ordinary surveying where the distinction in height above what? reaches a horizontal answer, and it is the reason a geoid model is part of a modern survey’s toolkit rather than a specialism.

The reduction in full, on one job #

It is worth doing the arithmetic once end to end, because the individual terms are easy and the order is not.

A ten-kilometre line, measured on the ground at 1,500 metres of ellipsoidal height, at 45° latitude, on UTM’s central meridian:

1. the measured distance: 10,000.000 m;
2. times the elevation factor $R/(R+h) = 0.99976$: 10,000.000 × 0.99976 = 9,997.647 m on the ellipsoid — the line has shrunk by 2.35 m;
3. times the grid factor 0.9996: 9,993.648 m on the grid — a further 4.00 m;
4. combined: 635 parts per million, or 6.35 metres between what the instrument read and what the coordinates say.

Six and a third metres on a ten-kilometre line, from two corrections neither of which anybody would call large. The same line on a low-distortion grid tuned to 1,500 metres comes out at 10,000.000 m and needs no reduction at all.

The order matters in one respect worth flagging: the elevation factor is applied first, because the grid factor is a property of a position on the ellipsoid rather than on the ground. Applying them the other way is wrong in the second decimal place of the correction, which is a millimetre here and is the same class of mistake as the datum transformation’s non-commuting inverse.

Where the model stops #

A single factor for a line is an approximation. The grid factor varies along a line, and the correct treatment integrates it — or, in practice, uses a weighted mean of the values at the ends and the middle. For a ten-kilometre line the difference is sub-millimetre; for a fifty-kilometre line in a zone’s outer parts it is not.

The elevation factor uses one radius for all directions, which is what the Gaussian mean radius is for. A more careful reduction uses the radius in the direction of the line, which differs from $\sqrt{MN}$ by up to the ratio of $M$ to $N$ — a few tenths of a per cent of the correction itself, which is parts in $10^9$ of the distance.

Nothing here is about the arc-to-chord correction, which is the other thing standing between a measured angle and a grid bearing. Distances need the combined factor; directions need the convergence and the arc-to-chord term, and those are grid north is not north.

Refraction and instrumental scale are excluded. An electronic distance meter has its own scale error and its readings depend on the atmosphere along the path. Those are calibration questions, they are of comparable size to the corrections here, and they are independent of them.

Why the two reductions are so easily confused #

They have the same units, the same order of magnitude and the same shape of correction, and they are about completely different things. Four differences worth keeping apart:

What they refer to. The elevation factor is about the shape of the Earth — two lines from the centre diverge, so a chord higher up is longer. The grid factor is about the map — a projection cannot preserve distance and does not.

What they depend on. The elevation factor depends only on height and latitude, and not at all on which grid is in use. The grid factor depends only on position within the grid, and not at all on the height of the ground.

Their signs. The elevation factor is always below one, without exception. The grid factor is below one near a zone’s centre line and above one outside its zero lines, so on a secant construction it takes both signs within one zone.

Who is responsible for them. The elevation factor is the surveyor’s, and cannot be designed away. The grid factor is the grid designer’s, and is precisely what a low-distortion projection designs.

The confusion has a consequence beyond bookkeeping. A practitioner who thinks of the two as one “scale correction” cannot see the cancellation, because the cancellation depends on one of them being adjustable and the other not.

The generalisation #

The portable form of this is about corrections that reinforce and corrections that cancel, and it is worth having as a habit rather than as a fact.

Two corrections with opposite signs have a point where they cancel, and that point is a design parameter. Two corrections with the same sign have no such point, and the only options are to make each smaller. Deciding which situation is in hand costs nothing and changes what can be done: a designer who knows the two terms oppose can choose the operating point, and one who does not will try to minimise each separately and get a worse answer than the combination allows.

The low-distortion grids are the clean example. A designer minimising projection distortion alone chooses a scale factor near one on the central meridian, which is what UTM does, and the surveyor at 1,500 metres then carries 635 parts per million. A designer minimising the combined factor chooses a scale factor above one, accepts worse projection distortion by the classical measure, and hands the surveyor a factor of one. The second grid is worse as a map and better as a tool, and which is wanted is a question about purpose rather than about mathematics.

The same shape occurs in what a standard parallel buys: one zero line with error growing away from it becomes two zero lines with error of both signs between them, and the worst case falls. Opposing errors are an opportunity; the mistake is to treat every correction as something to be individually minimised.

What a grid designed the other way looks like #

It is worth being concrete about the alternative, because “design the grid for the ground” sounds vaguer than it is.

The ground a low-distortion grid is designed for is a band four degrees across rather than a continent, and over that much latitude the curvature varies by 65 parts per million — which is the floor on how well any single scale factor can serve the whole of it.

The recipe has three numbers in it and each is chosen against the ground rather than against a mathematical criterion: a projection type, a central meridian near the middle of the area, and a scale factor set so that the combined factor is one at the area’s mean elevation. Everything after that is arithmetic.

What it buys is that a surveyor can compare a measured distance with a computed one directly, at any point in the zone, to within the residual variation — a few tens of parts per million rather than several hundred. What it costs is zone count, and zone count costs exactly what UTM and the zone system says it costs: boundaries, coordinates that carry no record of which zone they belong to, and jobs that straddle two.

The cancellation at another latitude #

There is a second correction a grid needs which this essay does not treat: the angle between grid north and true north across a zone. Distances take the combined factor; directions take that angle and the arc-to-chord term, and neither of those is a multiplier.

Naming what is left out is worth a sentence. A survey reduced to a grid needs both — the combined factor for every measured distance and the convergence for every measured direction — and they are independent, so getting one right says nothing about the other.

Who found it, and when #

The two reductions are as old as the idea of computing coordinates from measurements, which is to say they belong to the triangulation era. Neither is subtle, and both appear in nineteenth-century survey manuals in essentially their modern form.

What changed was which one dominated. In a triangulation, distance is measured once — a baseline, a few kilometres long, at a carefully chosen site — and everything else is angles. So the elevation factor applied to one line, at one place, and the grid factor was the term that propagated. With electronic distance measurement, from the 1960s, every line is a measured distance, and both reductions apply everywhere, every time. The corrections did not change; their frequency did.

The low-distortion projections are recent and are a North American speciality. The State Plane Coordinate System of 1927 used the classical design, with scale factors below one; the observation that a grid could be tuned to the ground’s elevation instead came out of practice in mountainous states in the 1990s, and the systems built on it multiplied through the 2000s. The 2022 modernisation of the United States’ reference systems includes a framework for them.

The vocabulary is unusually clear here, for once. Elevation factor, grid factor and combined factor each say what they do, and the only trap is the one this essay dwells on — that elevation in “elevation factor” means ellipsoidal height, and elevation everywhere else in surveying means the orthometric one.

Why the cancellation is a hazard rather than a convenience #

The two reductions very nearly cancel at the latitude and height this essay works at, and that is a pleasant arithmetical fact with an unpleasant consequence for how the pair gets checked.

A combined factor near unity looks like a job that needs no reduction. It is not: it is two corrections of a few hundred parts per million with opposite signs, each of which is present, each of which varies independently, and whose near-equality at one place says nothing about their near-equality at another.

The two vary with different things. The elevation factor depends on height above the ellipsoid and not at all on where the line sits in the zone; the grid factor depends on the distance from the central meridian and not at all on height. So a job that moves fifty kilometres east and stays at the same altitude changes one factor and not the other, and a job that climbs a thousand metres in the same place changes the other and not the one.

Which is why a spot check is worthless. Evaluating the combined factor at one station, finding it within a few parts per million of unity, and concluding that the reduction can be skipped is a conclusion about that station. The cancellation is a coincidence of one height and one easting, and the pair separates as soon as the work moves in either direction.

The check that does work is to compute both factors separately and look at their sizes, not their sum. Two terms of 300 parts per million that happen to cancel is a job that needs both reductions; two terms of 3 parts per million is a job that needs neither. The combined factor cannot distinguish those two situations and is the number most software reports.

Where this goes next #

That completes the vertical as it appears to a working surveyor. One thing has been assumed throughout: that changing a datum moves a coordinate sideways. It moves it up as well, by tens of metres, and the component is dropped so routinely that most software has nowhere to put it — which is the third coordinate moves too.