An angle is a difference, and the difference doubles the error

A bearing on a sphere is decided by its latitude, not its radius measured what happens when the ellipsoid is replaced by a sphere and a direction is taken on it. Four of the six substitutions are wrong at first order in the flattening, by an amount that runs as the sine of twice the azimuth; the conformal and geocentric spheres are wrong only at second order; and Bessel’s, which corrects the longitude as well as the latitude, is exact.

A surveyor does not measure one direction. A triangulation measures the three angles of a triangle, and an angle is a difference of two directions taken at one corner. That ought to help, and the reason is almost a reflex: a turn applied equally to every direction at a point vanishes from any difference of two of them, exactly, whatever its size. If a substituted sphere merely rotated the horizon, a triangulation would never notice it.

It does not merely rotate the horizon.

Why the reflex is a reasonable one #

The expectation that a difference cancels is not naive, and it is worth stating properly before it is taken apart, because it is correct about almost everything else a theodolite is subject to.

An instrument set over a point has an orientation that nobody knows: the zero of its horizontal circle points wherever it points. Every direction read from it is therefore wrong by one unknown amount, the same for all of them, and every angle formed as a difference of two such readings is exactly right. That is not a lucky cancellation but the whole design of the instrument — it is why a theodolite measures angles at all rather than azimuths, and why a network needs one observed azimuth somewhere and not one at every station.

The same argument disposes of several real errors. A tilt in the vertical axis, to first order, turns the horizon by a fixed amount. So does an error in the assumed orientation of the reference frame. A traverse must close leans on the same fact from the other side: a traverse accumulates the angles it turns through and never the directions it points in, so an unknown starting orientation costs it nothing but a final swing.

A substituted sphere looks like it belongs in that list and does not, and the difference is one word: the turn it applies is not fixed. It has a pattern, and the pattern has a period of a half-turn.

The law, and it has no fitted number in it #

Write the error a substitution makes in a direction at azimuth $\alpha$ as $k\sin 2\alpha$. The angle at a corner between directions $\alpha$ and $\alpha+s$ is wrong by the difference of the two errors, and that difference has a closed form:

$$\sin 2\alpha - \sin 2(\alpha+s) = -2\,\sin s\,\cos(2\alpha + s)$$

Turning the whole triangle on the spot sweeps $\alpha$ while holding $s$, so the largest angle error over all turnings is $2\lvert\sin s\rvert$ times $k$, which is the largest error in a single direction.

The measurement sits on that curve to within nine thousandths across a full sweep of the shape, and the two ends are worth naming separately because they are opposite answers to the same question.

At a half-turn the two directions sit on the same point of a pattern whose period is a half-turn, so the two errors are identical and the difference is nothing: the ratio measures 0.009, which is rounding. A corner whose two sides run in opposite directions — a point on a straight traverse — is immune.

At a quarter-turn the pattern reverses between the two directions, so the errors are equal and opposite and the difference is twice either: the ratio measures 2.000. And a surveyor’s triangle is nearer that end than the other. A well-conditioned triangulation avoids thin triangles precisely because they weaken the fix, so its corners sit between forty and a hundred and forty degrees — where $2\lvert\sin s\rvert$ runs from 1.29 to 1.29 with 2.00 in the middle.

So the reflex is exactly wrong. Taking a difference does not protect a triangulation from a substituted sphere; it is the geometry every triangulation is designed to have that makes the damage worst.

What each sphere costs a triangulation #

Five hundred and twenty-one arcseconds is eight and a half minutes of arc. A first-order theodolite reads to a tenth of a second and a triangulation is adjusted to a second or two, so a substitution that costs eight minutes is not a small error to be carried in a budget; it is a different survey.

The four first-order substitutions keep the ratios they had for a single direction — 1, ½, ⅓, ¼ for the geodetic, parametric, authalic and rectifying latitudes — because a difference of two proportional quantities is proportional in the same way. What changes is only the constant in front, and that is the $2\lvert\sin s\rvert$ above.

The conformal and geocentric spheres come through at one and a half arcseconds, which is the same order as the observations themselves and is therefore a real but arguable error. Only Bessel’s sphere is exact, and it is exact for the same reason it was exact for a single direction: it is not a latitude substitution at all but a change of both coordinates, chosen so that the geodesic of the ellipsoid becomes a great circle of the sphere. Four radii of the Earth is the neighbouring case where a substitution was chosen to be exact for one property and turned out approximate for the rest; this is the one substitution on which that trade does not arise, because it is not a substitution of one quantity.

The latitude dependence is the one comfort available and it is small comfort. The error falls as the square of the cosine of the latitude, so a survey in Norway is served twenty times better than one in Kenya — but at 80° north the geodetic-latitude sphere still carries 35 arcseconds into every angle, which is thirty times the observation noise.

What that is in the surveys that were built #

Three orders of triangulation have historically been run at three scales, and the measurements above price the substitution at each.

A first-order chain has sides of twenty to fifty kilometres, and the geodetic-latitude sphere costs it 521.1 arcseconds at 45° north on a thirty-kilometre triangle. A second-order network at ten kilometres pays 520.8. A third-order break-down at three kilometres pays 520.7. The numbers are the same to a part in a thousand because, as the next measurement shows, the error does not care about the size of the triangle at all — so the finer the survey, the worse the substitution is relative to what the survey is trying to achieve.

How big a triangle it takes measured what a closing error of a surveyed triangle is worth as a reading of the instrument’s own noise, at these same sizes; how many triangles it takes counted what a chain of them accumulates. Neither of those quantities is what is being priced here. They are about observation, and this is about the sphere the observations are computed on, which is a decision taken once and applied to all of them.

The error does not shrink with the triangle #

This is the part with no analogue anywhere else in the subject. Almost every error measured against the ellipsoid gets smaller as the region does — that is what makes a local survey easier than a continental one, and it is the assumption behind every plane-survey approximation there has ever been.

A latitude substitution does not get smaller. On a one-kilometre triangle the geodetic-latitude sphere is out by 601 arcseconds and on a four-hundred-kilometre triangle by 563, which is the same number. The reason is that the error is a property of the corner and not of the journey: the direction the geodesic leaves on is wrong by $k\sin 2\alpha$ whether it is going one kilometre or four hundred, because what is wrong is the latitude the corner is placed at.

The conformal sphere behaves the way one expects, growing in proportion to the side. So does the excess, growing as the square. It is the first-order substitutions that refuse to shrink, and shrinking the survey is the only remedy most errors in this subject have.

The closure check is blind to all of it #

The check a surveyor actually runs on a measured triangle is its closure: the three angles should sum to two right angles plus the spherical excess, and a departure is a misclosure to be investigated and then distributed.

That check cannot see any of this. The sum of the three angles is reproduced by every substitution to a small fraction of what it does to the individual angles, and by the geodetic-latitude sphere to two parts in a thousand million — while each of its three angles is out by minutes. A triangulation computed on the wrong sphere closes exactly as well as one computed on the right one.

The mechanism is visible in the arithmetic if not fully explained by it. Each of the six directed azimuths round a triangle appears once in the sum of the three interior angles, with a sign, and the sines of twice those azimuths very nearly cancel in that combination. Why the cancellation is exact to rounding for the geodetic latitude and leaves a small residue for the others — six thousandths of an arcsecond at thirty kilometres, three tenths at two hundred — is not established here. That it holds is measured; why it holds for one substitution and not quite for the rest is not.

This is the same shape as the defect the blunder the network cannot see prices for a gross error in one observation: what an adjustment absorbs into its coordinates is exactly what its residuals cannot report, and the two shares add to one. There the split is set by the redundancy of each observation; here it is total, because the error is in the model the adjustment is computed with rather than in anything the adjustment is weighing. A coordinate is the output of a solve is the standing warning, and this is its sharpest instance: the solve is working correctly on the wrong surface, and nothing internal to it can say so.

What a closed figure cannot see is the general statement of this trap on the survey side: a misclosure detects the errors a closure is sensitive to, and a systematic error that leaves the closure alone is invisible to it however large. Here the systematic error is not in the observations at all. It is in the computation, and the check that would catch a bad observation is constructed so as to pass a bad ellipsoid.

The one substitution worth a second look #

The conformal sphere deserves separating out, because it is the substitution a cartographer would reach for and its behaviour here is different in kind rather than in degree.

Its error at 30 km is 1.57 arcseconds, which on a first-order triangulation is comparable with the observations rather than dwarfing them — and it grows with the triangle, proportionally, so a survey that shrinks its triangles buys real relief from it. That is the ordinary behaviour of an approximation, and it means the conformal sphere can be used inside a stated tolerance by keeping the triangles small enough.

The four first-order substitutions cannot be used that way at all. Their error is the same on a three-kilometre triangle as on a fifty-kilometre one, so there is no tolerance a surveyor can buy by working finer, and no size below which the substitution becomes harmless. An ellipsoid computed to a nanometre is known to a decimetre is about a different kind of budget — one where the arithmetic is far better than the knowledge behind it — and this is the opposite case: the arithmetic is exactly as good as the surface it is done on, and the surface is wrong by a fixed amount wherever it is used.

What each number was checked against #

On a sphere every substitution must be exact. Run on a genuine sphere, all six must keep every angle to a millionth of an arcsecond, and do. Without this the whole essay is unfalsifiable, since any difference of two integrated directions is overwhelmingly likely to be non-zero.

Bessel’s sphere must keep an angle as it keeps a direction, and is out by 6 × 10⁻⁹ arcseconds on the stated triangle.

Every other sphere must make a real angle error, so that the exactness above is a property of Bessel’s construction rather than of a test too coarse to see anything.

The law must have no fitted parameter. At every shape from five degrees to a hundred and eighty, the largest angle error over all turnings, divided by the largest single-direction error, must equal $2\lvert\sin s\rvert$ to within 0.06. The measured worst departure is 0.009.

Both ends of it must be checked separately, because the expectation this measurement was made to test was that a difference always cancels: at a half-turn the ratio must be below 0.05 and is 0.009, and at a quarter-turn it must be between 1.9 and 2.1 and is 2.000.

Turning the triangle must move the error, since a turn that did not depend on direction is the only thing a difference could remove entirely; it runs 517.74″ to 604.26″ over a full revolution.

And the closure must be blind. For every one of the six substitutions the error in the sum of the three angles must be under a hundredth of the worst error in any one of them, and for the geodetic latitude under a ten-millionth of an arcsecond.

What one triangle does not settle #

The triangles are equal-sided and stated. Every measurement here takes two sides of one length from one corner at a stated separation. A real triangulation has triangles of many shapes chained together, and the worst-case law above is a statement about one corner rather than about a chain.

The truth is Vincenty’s solution on WGS84. It is exact to a fraction of a millimetre over these distances and its azimuth to well under a thousandth of an arcsecond, so it is the truth here in the same sense it is elsewhere in this subject — but it is a truth about an ellipsoid, and the ground is not one.

The angle is the difference of two initial azimuths. A theodolite reads the horizontal angle between two observed directions, which involves the deflection of the vertical, the height of the targets and the atmosphere. None of those is here, and each is a real term in a real triangulation.

And the sum’s cancellation is measured, not derived. The essay states it and says plainly that it does not explain it. A mechanism that predicted the small residues for the other five substitutions would be worth more than the observation.

Still open: whether the survey that mattered was ever computed this way #

The numbers above are large enough to be alarming and the alarm may be misdirected, because they price a computation nobody may have performed. Legendre’s theorem and the classical reduction methods were built on an auxiliary sphere chosen with some care, and the national triangulations of the nineteenth century were computed by people who knew precisely which sphere they were using and why.

What is not settled here is which substitution a given historical computation actually made, and how much of the eight minutes above would have reached the published coordinates. A triangulation adjusted to close will absorb a systematic angle error into its coordinates rather than reporting it — that is the whole of the previous section — so the error would appear not as a misclosure but as a slow distortion of the network, of a kind a later re-observation would find and attribute to something else. Whether any such distortion is visible in a surviving network, what size it would be at the scale of a country, and whether the pattern it makes is distinguishable from the other systematic errors those networks carry, are questions one triangle cannot ask.