The tolerance decides the model

A specification says apply the grid scale factor to lines over ten kilometres. Where does ten kilometres come from?

It comes from somebody inverting a correction against a tolerance, and it is right for the tolerance and the site they had in mind. Both are usually missing from the specification, and a rule with its conditions stripped off is indistinguishable, in a sentence, from a measurement.

Inverting each correction #

Three of the four corrections are proportional to the line’s length, so each inverts directly.

The grid scale factor contributes $|k - 1|$ of the length. At 393 parts per million that reaches a centimetre at 25 metres.

The elevation factor contributes $h/R$. At 150 metres up that is 23.5 parts per million, reaching a centimetre at 426 metres.

The slope reduction contributes $1 - \cos\alpha$. At 2° that is 609 parts per million, reaching a centimetre at 16 metres.

Treating the patch as flat is different in kind. Its error goes as $r^3/6R^2$ for the chord against the arc, so it is cubic in the patch size rather than linear in the line’s length, and inverting it gives a radius of 13.5 kilometres.

The expectation that was wrong, again #

The essay was planned around the claim that a tighter tolerance reorders the corrections — that a millimetre job has different priorities from a decimetre job. The measurement refused it.

Three of the four rows are linear in the tolerance, so halving the tolerance halves all three lengths and leaves their order exactly as it was. Only the flat-earth row moves relative to them, and it goes as the cube root, so it moves slowly: the ranking is unchanged from a millimetre to two decimetres, a span of two hundred to one.

That is a better result than the one expected. A specification that ranks the corrections for one tolerance has ranked them for every tolerance, so the ranking can be published once and used everywhere, which is exactly what a specification wants to be able to do.

What does reorder them #

The site.

At the same centimetre tolerance, a job 2,200 metres up on gentle ground and a job at sea level on 6° of slope rank the same four corrections differently. On the mountain the elevation factor is the tightest constraint; at the coast it is the slope reduction, by a wide margin, and the elevation factor has fallen to fourth.

That is the assertion the machinery carries, and it is stated as a pair. The ranking must be the same at a millimetre as at two decimetres, and it must differ between two jobs at the same tolerance. Without the first half, the second could be satisfied by a table whose rows were unstable for any reason at all.

So the honest form of a specification’s rule is not apply this correction over ten kilometres but apply this correction over ten kilometres, at this tolerance, at this height, on this slope, at this distance from the meridian — and the last three are what get dropped.

The two jobs, side by side #

The site-dependence is worth setting out in full, because it is the essay’s operative finding and a single sentence undersells it.

A mountain job at 2,200 metres above the ellipsoid, on 0.4° of ground, on the central meridian. The elevation factor is 345 parts per million and reaches a centimetre at 29 metres. The slope reduction is 24 parts per million and reaches a centimetre at 410 metres. The grid factor is unchanged at 393 parts per million, at 25 metres.

A coastal job at 10 metres, on 6° of ground, three and a half degrees off the meridian. The elevation factor is 1.6 parts per million and reaches a centimetre at six kilometres. The slope reduction is 5,483 parts per million and reaches a centimetre at under two metres. The grid factor has moved too, because the point is far from the meridian.

Same tolerance, same four corrections, and the tightest constraint has moved from one row to another while a row that was second has fallen to last. A specification’s list, transferred from one to the other, is not merely imprecise — it directs attention at the wrong correction.

The two orderings, and why they differ #

Comparing the two figures is the point of having both.

By size on a given line, the ranking is slope, then grid, then elevation. By length at which it first matters, it is slope, then grid, then elevation — the same, because all three are linear and a linear correction’s size and its threshold are reciprocal.

The flat-earth term is where they come apart. It contributes nothing at all to a line’s reduction — it is not a multiplier — and it has a threshold, because beyond a certain patch size the plane assumption itself fails. It appears in one figure and not the other, and a practitioner who has only the first has no way to know the second exists.

That is the structural reason a decision table is worth computing separately from an error budget. An error budget lists the terms that are wrong; a decision table lists the assumptions that stop holding, and the second list is longer.

What a specification could say instead #

The finding suggests a form a specification could take that would be both shorter and correct.

Publish the four corrections’ parts per million as functions of the job’s own parameters — the height, the slope, the distance from the central meridian — and let the reader invert them against their own tolerance. That is three formulae and a sentence, it is right at every tolerance and on every site, and it replaces a table of thresholds that is right on one.

The reason specifications do not do this is that a formula requires the reader to compute and a threshold does not. That was a real constraint when computing meant a slide rule and it stopped being one decades ago, which puts survey specifications in the same position as the survey manuals’ three-point Simpson’s rule that the scale factor of a line examines: a convention shaped by an arithmetic cost that has gone.

The difference is that Simpson’s rule is exact for its integrand, so nothing is lost by keeping it. A threshold rule is exact for one site, so a great deal is.

The rate that distinguishes the rows #

The claim that three rows are linear and one is not can be checked without trusting the algebra, by measuring how each threshold moves when the tolerance changes.

Multiply the tolerance by eight. The grid scale factor’s threshold multiplies by 8.0000. The flat-earth radius multiplies by 2.0000, which is the cube root of eight.

Two different powers, measured rather than asserted, which is what pins down that the rows are genuinely different shapes rather than differently-scaled versions of one shape. It is the same discipline as the fourth-order convergence in the scale factor of a line: a rate distinguishes a mechanism from a coincidence, and a magnitude does not.

The corrections this table does not have a row for #

Four rows is not the whole list, and the omissions are instructive because each was left out for a different reason.

The height mistake — using a levelled height where an ellipsoidal one is wanted — is not a correction but an error, so it has no threshold in the sense the others do. Its size is $N/R$, about 7.6 parts per million in Britain, which would put it between the elevation factor and the grid factor if it were a row. What a tape measures puts a number on it: 583 millimetres on a 77-kilometre line, or thirty times a survey’s own closing tolerance.

The unit has no threshold either, because it is not proportional to the line at all — it is proportional to the coordinate, which is the whole point of the units are part of the coordinate. A table of line lengths cannot express an error that does not depend on line length.

The datum is off the scale entirely at a hundred metres, and belongs to a question upstream of every row here: whether the coordinate system is the right one before any correction is applied to anything.

The pattern is worth naming. A decision table can only rank the errors that share its independent variable, and three of the most important failures in this field do not. That is not a flaw in the table; it is a reason to have more than one, and to know which question each answers.

Where the flat-earth row comes from #

The last row is the only one that is not about a coordinate system, and it is the one with the deepest provenance.

The error of treating a patch of the Earth as flat grows as the square of its size — that is the quadratic law derived in how small is flat enough, and it is a consequence of Gaussian curvature rather than of any projection. Inverting it gives the radius at which a stated tolerance is reached, and it is the number that separates plane surveying from geodetic surveying.

Thirteen and a half kilometres at a centimetre. That is why plane surveying works at all, and why a national mapping agency cannot use it.

Everything else in the decision table is a property of a chosen coordinate system. This row is a property of the Earth, and it is the one row no choice of grid, unit or datum can move.

Two of the four rows are also not independent. The grid factor and the elevation factor multiply, and on a grid whose scale factor exceeds one they pull opposite ways and cancel exactly at one elevation — so at 2,551 metres both rows are wrong in the same direction and the table’s two entries have become one. The arithmetic is the ground is not the grid.

Where a tolerance comes from #

The table takes a tolerance as given, and it is worth asking where a tolerance comes from, because the answer is not measurement either.

A tolerance is a statement about what the work is for. A topographic map at 1:25,000 is drawn with a pen whose line is a quarter of a millimetre wide, which is six metres on the ground, so a coordinate good to a metre is far better than the medium can show. A boundary is held to whatever the jurisdiction’s regulation says, which is typically a few centimetres and is a legal number rather than a physical one. A tunnel drive from both ends is held to whatever the two headings must agree to when they meet.

None of those is derived from an instrument’s capability. They are derived from purpose, and the instrument is then chosen to meet them — which is the right order and is the opposite of how a tolerance is usually discussed.

That is the collection’s standing rule appearing in a new place. No essay here may say a projection is best without naming the purpose, and a decision table is the same claim in tabular form: it says which corrections matter, and it cannot say anything at all until somebody supplies what the work is for.

What was computed, and how #

Each linear correction’s parts per million, from the projection’s own derivatives for the grid factor, from the Gaussian mean radius for the elevation factor, and from the cosine for the slope.

Each threshold, by inverting the correction’s own expression against the stated tolerance rather than by scanning — so the numbers are exact given the model rather than being sampled.

The flat-earth radius, by inverting the cubic.

Four assertions. The ranking must be tolerance-invariant; it must be site-dependent; the flat-earth row’s threshold must scale as the cube root; and the grid row’s must scale linearly. The last two are what prove the rows are different shapes.

Where the model stops #

Each correction is considered alone. A real job carries all four at once, and their combination is not the minimum of the four thresholds — errors of the same sign add. A table of individual thresholds is a screening device, not a budget, and using it as a budget is optimistic by a factor of about two.

The corrections are treated as independent. The grid scale factor and the elevation factor are not: their product is the combined factor, and on a grid whose scale factor exceeds one they partially cancel, which is the whole subject of the ground is not the grid. At the elevation where they cancel exactly, both rows of this table are wrong in the same direction.

And nothing here is about the instrument. Every threshold above assumes the observation itself is perfect, which is the assumption a traverse must close shows a closure cannot check. A real job’s error budget starts with pointing, centring and the instrument’s own specification, and those are frequently the largest terms of all — which means the table’s tightest row may not be the job’s tightest constraint.

The generalisation #

Inverting a correction against a tolerance turns a quantity into a decision, and the resulting ranking is more stable than anybody expects.

The second half of that is the finding. The instinct is that a tighter requirement changes what matters, and it does not, provided the corrections are all linear in the same variable — because scaling the requirement scales every threshold by the same factor. Where a ranking does change, it is because the terms have different functional forms, and the flat-earth row is the only one here that does.

The practical consequence is a good one for specification writers: publish the ranking, publish the conditions it was computed under, and let a reader rescale it. The ranking survives rescaling. The conditions do not, and they are the part that gets left out.

Who wrote the rules, and when #

Threshold rules of this shape are as old as survey specifications and their provenance is almost always lost. Apply the scale factor over ten kilometres, sea-level correction above 500 metres, plane methods within 20 kilometres — each was computed once, by somebody, against a tolerance and a site, and then travelled without them.

The travelling is what makes them look arbitrary. A rule of thumb whose conditions are known is a decision table; the same rule with the conditions stripped is folklore, and it is defended and attacked with equal confidence by people who have no way to check it.

Recomputing them is not difficult, which is the mildly embarrassing part. Every number in this essay comes from inverting an expression that has been in the textbooks for a century, and the reason the rules are quoted rather than recomputed is that quoting is faster and nobody is checking — the exact circumstance this collection’s founding rule was written against.

A rule with its conditions attached is a different object #

The observation that a threshold rule travels without its tolerance and its site is the essay’s most portable finding, and it is worth stating what the repair looks like, because it is not stop using rules of thumb.

The rules are good. Apply the scale factor over ten kilometres is a correct summary of an inversion somebody performed, and a practitioner following it gets the right answer for the case it was computed for. Nothing about the practice of compressing a calculation into a threshold is wrong; it is what a specification is for.

What is missing is two numbers and a place. The tolerance the rule was written against, and the conditions — the latitude, the height, the zone position — at which the inversion was performed. With those attached, the rule is a decision table entry that a reader can recompute for their own case. Without them it is a number with no derivation, which can be neither checked nor adapted.

And the difference shows up exactly when somebody works outside the original conditions, which is the case the bare rule gives no warning about. A threshold computed for a temperate national grid applied at high latitude, or at altitude, or near a zone edge, is being applied outside its derivation with nothing to say so.

The cost of attaching them is a parenthesis. Apply the scale factor over ten kilometres (for 10 mm, at 50°, mid-zone) carries the whole derivation in eight words and lets a reader with a different tolerance invert it themselves. That is the entire proposal, and it is what separates a decision table from folklore.

Where this goes next #

Every threshold here is measured against coordinates assumed correct. The coordinates a job starts from are published control values, and those turn out not to be observations at all: a published coordinate is a result.