How small is flat enough

Every practical use of a map begins by ignoring this site’s founding theorem. A site plan is drawn in plane coordinates, distances are computed with Pythagoras, and nobody is wrong to do it.

No map is faithful is a theorem, and it is not a prohibition. It says the error cannot be zero; it says nothing about how large it is, and the useful question is the second one.

The bound, and where it comes from #

The quantity that matters is the ratio between the largest and smallest scale factor over the patch — if a map holds every distance to within a factor of $1 + \epsilon$, then $\epsilon$ is what a survey has to live with.

For a circular patch of angular radius $\rho$ on a sphere, the smallest achievable value has a closed form. It is Chebyshev’s criterion in the one case where the answer is known exactly: the optimum is the stereographic projection centred on the patch, and its scale spread over a cap of radius $\rho$ is

$$\frac{k_{\max}}{k_{\min}} = \sec^2\!\left(\frac{\rho}{2}\right)$$

No projection of that cap does better, conformal or otherwise. Expanding for small $\rho$,

$$\sec^2\!\left(\frac{\rho}{2}\right) - 1 \;\approx\; \frac{\rho^2}{4} \;=\; \frac{s^2}{4R^2}$$

with $s = R\rho$ the radius on the ground. The unavoidable error is quadratic in the size of the patch, with a coefficient fixed by nothing but the radius of the Earth.

Why quadratic is the surprising part #

A linear law would make the boundary between plane and curved work a matter of taste — there would be no size at which the situation changes character, only a slow deterioration.

A quadratic law makes the transition sharp in both directions. Ten times larger is a hundred times worse; ten times smaller is a hundred times better. A patch of 1 km radius has an unavoidable error of 0.006 parts per million, which is six nanometres per kilometre and is far below anything measurable. A patch of 300 km radius has 555 ppm, which is half a metre per kilometre and is a scandal. Between them lies a range of about one and a half decades in size over which the answer goes from obviously fine to obviously not, and inside that range the question has to be asked rather than assumed.

The measurement, and the exponent as the check #

The table above is a closed form, and a closed form written down is a closed form nobody checked. It is measured here by a second route: build the stereographic projection centred on the cap, sample its scale factor over the cap with the site’s ordinary machinery, and take the ratio of the extremes.

The two agree to better than one part in ten thousand from 10 km upwards. Below that the measured value runs high — 0.030 ppm against 0.006 at 1 km radius — because the sampler’s own arithmetic noise has become comparable to the quantity being measured, which is a fact about the measurement and is reported as one rather than hidden by starting the range at 10 km.

What is asserted on every build is the exponent, fitted across six decades: 2.0000. That is a much stronger test than a tolerance on any single value. An error in the conversion between angle and distance, or in the radius used, would shift the coefficient and leave the exponent untouched; an error in the geometry — taking a diameter for a radius, say — would move both. Only the right law gives exactly two.

The refusal is the law someone would guess without doing the expansion: a linear one. A fitted exponent of 1 fails.

Inverting it #

The useful direction is the other one: given a tolerance, how large may the patch be?

$$\rho = 2\arccos\frac{1}{\sqrt{1 + \epsilon}}$$

which for the tolerances an actual survey works to gives

Ten parts per million is a millimetre in a hundred metres, which is around the accuracy of good ordinary work with a total station, and it buys a patch 81 km across. That is roughly an English county, and it is the reason a county-sized project can be run on a local plane grid while a national one cannot.

The inversion is checked by round trip: the radius returned for 10 ppm is put back through the forward calculation and must come out at 10 ppm to within $10^{-6}$.

The same law at a surveyor’s tolerance #

Redrawing the figure at one part per million rather than ten moves the crossing point but not the line, which is the point of a law with a fixed exponent.

That square-root relation between tolerance and size is worth carrying around. It says that the enormous improvement in measurement accuracy over the last century — from parts per thousand to parts per million and beyond — has shrunk the patch that can be treated as flat by a factor of about thirty rather than by a factor of a thousand. Plane surveying survived the arrival of electronic distance measurement for that reason.

What a national grid does about it #

A country larger than 81 km across has three options and uses all of them.

Accept more distortion. UTM’s zones are six degrees wide, which at the equator is 667 km across, and the resulting scale error is 981 ppm after the 0.9996 constant has done its work. That is a hundred times the tolerance above, and it is accepted because a UTM coordinate is a national or global reference rather than a set of site dimensions: the distortion is known and can be corrected for, which is different from being small.

Use more zones. Sixty of them, in UTM’s case, at the cost of a discontinuity at every boundary.

Correct the arithmetic instead of the map. A survey computed on a grid applies a line scale factor to every measured distance and an arc-to-chord correction to every measured angle, at which point the grid’s distortion stops being an error and becomes a systematic transformation with a known inverse. That is what makes a projected coordinate system usable at any size, and it is a different activity from choosing a projection small enough not to need it.

The same trade at zone scale is two orders of magnitude larger. Across a six-degree UTM zone the tangent construction is 1,382 parts per million out at the edge and the scaled one 981 — both far past the ten parts per million a plane survey works to, which is why grid work corrects rather than ignores.

The angular version of the same limit #

Scale is one of the quantities a plane survey gets wrong. The other is the angle sum of a triangle, and it has its own quadratic law.

The spherical excess of a triangle is its area divided by $R^2$, exactly — that is Gauss–Bonnet on a sphere — so the angles of a triangle on the ground sum to more than 180° by an amount proportional to the area it encloses:

A theodolite reading to one arcsecond detects the curvature of the Earth in a triangle about 20 km on a side, which is the same order as the scale limit and for the same underlying reason: both quantities are the curvature multiplied by an area.

That coincidence of scale is worth stating plainly, because it means the two failures of plane surveying arrive together. There is no regime in which a survey’s distances are fine and its angles are not, or the reverse.

Which projection is nearly optimal, and by how much #

The bound belongs to the stereographic projection centred on the patch, and no real survey uses that. What the alternatives cost over a small region is measurable, and the answer is reassuring in a way the world-map figures never are.

This is the practical counterpart of the quadratic law. Over a large region the choice of projection dominates everything — Mercator against Gall–Peters is an argument about a factor of fifteen — and over a small one it barely matters at all, because every smooth map is a similarity to first order and the differences between them appear at the same second order as the unavoidable error itself. A survey worrying about which conformal projection to use over a 10 km site is worrying about the wrong term.

The bound is a floor, not an estimate #

Two things follow from the bound being a bound rather than a typical value.

No projection does better. The 0.62 ppm over a 10 km patch is not what a well-chosen projection achieves; it is what nothing beats. A survey using the national grid rather than a local optimum will do worse, often by a large factor, because the national grid is optimised for the nation and the patch is not at its best point.

Doing better means abandoning the plane. Or the sphere: everything here is a statement about mapping a sphere, and the projection whose property is measured against the wrong body is a reminder that the assumed surface is itself a choice with a size.

To be precise about the alternative: The only way past the floor is to stop making a flat map and compute on the ellipsoid directly, which is what geodetic computation is: distances by Vincenty’s formulae or by integrating the geodesic equation, angles as azimuths on the reference surface. That is more work and it has no size limit at all.

The patch is not where a survey thinks it is #

One further correction turns the bound from a statement about geometry into a statement about a job, and it is the one most often skipped.

The bound is for a map centred on the patch. A survey working in a national grid is not at the centre of anything: it is somewhere in a zone whose centre is elsewhere, and the distortion it inherits is the grid’s value at its location, which can be an order of magnitude worse than the best available and is not quadratic in the site’s own size at all. It is quadratic in the distance from the grid’s line of true scale.

That is why a site plan drawn on a national grid and a site plan drawn on its own local grid disagree, and why the disagreement does not shrink as the site does. Shrinking the site removes the site’s own curvature error and leaves the grid’s offset untouched — a systematic scale factor of a few hundred parts per million, uniform across the whole job, which then propagates into every quantity computed from the coordinates.

The remedy is the one every survey manual gives and few explain: compute a local scale factor at the site’s own position and apply it. That converts the grid’s distortion from an error into a known constant, and returns the residual to the quadratic law, at the site’s own size.

Where the folk rules come from #

Surveying practice carries several rules of thumb about when curvature matters, and the law above explains where each of them comes from.

“Under 10 km, ignore it.” At 10 km radius the unavoidable scale error is 0.62 ppm, or 6 mm in 10 km, comfortably below ordinary measurement error.

“Under 20 km, ignore the angles.” The excess of a 20 km triangle is about 0.9 arcseconds, which is the reading precision of a good instrument.

“A local grid is good for a city, not a county.” A city is 20–40 km across, giving under 2 ppm; an English county is 80–100 km, arriving exactly at the 10 ppm figure.

None of these is a convention. Each is the quadratic law evaluated at the accuracy of the instruments of the period in which the rule was coined, which is why the numbers have drifted downwards as instruments improved: a rule written for chain surveying tolerates a patch several times larger than one written for satellite positioning.

The scale error is one of the two things a plane survey gets wrong. The other is a direction, and it has the same threshold for the same reason.

The second failure disappears at the same moment the first does. The arc-to-chord correction on a seventeen-kilometre line at 52° north is under an arcsecond everywhere in the zone, which is the reading precision of a good theodolite.

What was computed here #

The closed-form bound $\sec^2(\rho/2)$ was evaluated at eight patch radii from 0.5 km to 1,000 km, and independently measured by sampling the scale factor of the stereographic projection centred on each cap over sixteen equal-area rings, taking the ratio of the extreme values.

The exponent was fitted by least squares in log–log space across six decades and must come out within 0.01 of 2, with a linear law as the refusal. The measured spread must agree with the closed form to within 5% for every patch of 10 km or more; below that the measurement is dominated by its own noise and is excluded, with the reason recorded rather than the range quietly trimmed.

The inversion is checked by round trip. The spherical excesses are area over $R^2$ with the areas computed in closed form.

What the pictures cannot show #

The bound is for a circular patch. A long thin region of the same area does better — its worst point is closer to its centre — and a region with a re-entrant boundary can do worse. The circle is the case with a closed form, and every real survey area is some other shape.

Nor does the figure distinguish the two ways a survey can fail. It plots the scale spread, which is a statement about distances; the angular failure has the same law and a different constant, and is in the table rather than the drawing.

The other reduction with the same law #

One other reduction obeys the same law and pulls the other way. Bringing a measurement down from the ground to the ellipsoid is a scale correction linear in the height — 157 parts per million per kilometre, always in the same direction — so set against a grid factor above one it cancels exactly at a stated elevation.

The comparison is instructive because the two errors behave differently in the same units. The plane-survey error is quadratic in the size of the patch, so it can be made arbitrarily small by working smaller. The elevation reduction is linear in the height and does not shrink with the patch at all — a hundred-metre baseline at two kilometres of altitude carries the same 314 parts per million as a hundred-kilometre one. The ground is not the grid follows it through.

The same bound for a drawn line #

The plane-survey bound is quadratic in the patch size and this essay inverts it for a stated tolerance. A line drawn between two stored points obeys the same law for the same reason, and the applied field fits the exponent rather than assuming it.

The departure of a straight segment from the ground it claims goes as κL²/8; measured over six lengths from 50 to 1,600 kilometres, the fitted exponent is 2.001. Inverted, a tolerance costs vertices as a reciprocal square root: eight pieces at ten kilometres, 128 at a hundred metres, 512 at ten. The same arithmetic that says a survey may treat 45 kilometres as flat says a stored segment may be straight for about the same distance.

The same bound holds for a drawn line rather than a survey, and the fitted exponent is a test of the measurement as much as of the geometry: anything away from two would mean the projection’s own distortion was being picked up. At survey scale it reads sixty kilometres of stored line lying 81 metres from the ground — below the resolution of any picture, and far above the tolerance the same ground would be surveyed to.

The bound is a maximum, and a job may not need it #

One refinement that costs nothing and is worth stating, because it changes the permitted patch by a fifth.

The quantity bounded above is the departure at the middle of the span, which is where a parabolic sagitta is largest. A job whose tolerance applies to the whole line — a stored geometry whose average error matters, a reduction whose residuals are aggregated — is not bounded by the midspan value but by the mean of it, and for a parabola the mean over the span is exactly two-thirds of the maximum.

So a tolerance stated on the mean permits a patch $\sqrt{3/2} = 1.22$ times larger than the same number stated on the maximum: 55 kilometres where the worst-case rule gives 45. That is not a large factor and it is not nothing, and the point of computing it is that the two rules differ by a fixed constant rather than by an exponent — so the choice of statistic never changes the shape of the law, only where its line sits.

Who found it, and when #

The practical rule is older than its explanation. Roman surveyors worked in plane coordinates over distances where the question does not arise; the mediaeval Islamic geodesists computing the direction of Mecca could not, and knew it.

The closed form comes out of Chebyshev’s 1856 criterion, proved by Grave in 1896. What made the quadratic law explicit as an engineering criterion was the arrival of national triangulation in the eighteenth and nineteenth centuries, where the two regimes met in one organisation: the same instrument that fixed a parish boundary on a plane had to be reconciled with an arc measurement that could not be.

Where this goes next #

Everything above is stated for a sphere of one radius. The Earth is not a sphere, and its curvature is not a single number either — which turns out to put a floor under the spherical approximation itself.