Where a fit leaves residuals

Seven parameters relate two Cartesian frames: three translations, three rotations, one scale. Applied to a rigid body of any size, they are exact — the transformation has precisely the degrees of freedom the problem has, and the fit closes to arithmetic noise.

A datum’s realisation is not a rigid body.

The claim #

A triangulation network is a set of published coordinates — the realisation that what a coordinate refers to names as the third of a datum’s four commitments — and those coordinates carry the accumulated error of the survey that produced them. That error is not random from mark to mark: adjacent marks were observed from each other, so their errors are correlated, and the correlation extends along chains for hundreds of kilometres. What comes out is a strain field — a slowly varying stretch and shear across the country, of a few parts per million.

The claim is that a seven-parameter transformation cannot absorb it, that the amount it cannot absorb is metres, and that what is left has a shape.

What was computed, and how #

The measurement needs a network with a known amount of strain in it, so the strain is stated rather than taken from a dataset. The rule is written down and is the following.

Take a grid of markers over Britain. Transform each one from the Airy ellipsoid to WGS84 with the published seven parameters — that is the truth, exactly. Then displace each transformed marker by a smooth field with a stated amplitude in parts per million, quadratic in position:

$$(\varepsilon_E, \varepsilon_N) = A\left(u^2 - v^2,\; 2uv\right), \qquad u = \frac{\lambda - \lambda_0}{L}, \quad v = \frac{\varphi - \varphi_0}{L}$$

Quadratic, deliberately. A constant strain would be absorbed by the scale parameter, and a linear one by the rotations, so the leading term a similarity transformation cannot reach is the second. Anything lower would be measuring the model’s ability to fit rather than its inability.

Then fit the seven parameters back by least squares over the displaced markers and look at what is left. The fit is linear in all seven — the rotations are of order a microradian, so the second-order terms are tenths of a millimetre — so it is one 7×7 normal-equation solve and there is no starting guess to get wrong.

The result: 1.64 metres of RMS residual, 3.00 metres at the worst marker, for four parts per million of strain.

The control, and why it is the important figure #

A residual of 1.64 metres means nothing until it is known what the machinery reports when there is nothing to report.

Three orders of magnitude and a bit. The method’s own error is 0.29 millimetres and the effect being measured is 1.64 metres, so the 1.64 metres is the strain and not the arithmetic.

Better than that: the 0.29 millimetres is identifiable. The linear model drops the term in scale times rotation, which for OSGB36’s parameters is $2 \times 10^{-5} \times 4 \times 10^{-6} \times 6.4 \times 10^{6}$ metres — about half a millimetre, which is what came out. Nothing unaccounted for remains. That is the difference between a control that says “small” and a control that says “exactly this, for this reason”.

The residual is a field, not noise #

The arrows in the hero figure are smooth. That is the second half of the claim and it needed its own measurement, because “smooth” is an impression and impressions are what this site exists to replace.

The first attempt averaged the easting residual over a quadrant of the grid, on the reasoning that noise averages away over thirty-six markers and a field does not. It passed at four millimetres against an RMS of 1.6 metres — that is, it reported no structure at all — and the check was wrong rather than the claim. The strain field used here is the real part of a square, which is antisymmetric under swapping the two axes, so it averages to nothing over any square patch. Averaging is the wrong instrument for shape.

What separates a field from noise is that neighbours agree. So the measurement became: the mean displacement between markers that are adjacent on the grid, against the same quantity for the same residuals shuffled into different positions. Identical values, identical RMS, no structure. The field’s neighbour step is less than half the shuffled one, and the control is what makes the comparison independent of how densely the markers were laid out.

That matters practically. A residual that is noise can be reduced by observing more marks. A residual that is a field cannot: more markers give a better picture of the same distortion.

Where the markers are decides what the parameters come out as #

This was going to be a one-line remark, and the arithmetic refused it.

The intended claim was that a fit absorbs part of the strain into its scale parameter, so the fitted parameters are not the true ones. Over the symmetric grid that is false: the scale comes back correct to eight decimal places in parts per million. The reason is orthogonality — the strain is quadratic, the seven parameters span the constant and linear parts, and over a symmetric layout the quadratic is orthogonal to all of them. None of it is absorbed and all of it lands in the residual.

Cut the layout to a wedge and the orthogonality goes with it. The fitted scale moves by 1.44 parts per million — nine metres on the ground — the translations move by metres, and the residual falls, because some of the strain is now hiding inside the transformation.

That is the mechanism behind a fact that otherwise looks like sloppiness: two authorities publish different seven-parameter sets for the same pair of datums, and both are correct. They fitted over different sets of stations. Neither set is the truth, because there is no truth of that shape to be had; each is the best rigid approximation to something that is not rigid, over the ground it was asked about.

The contamination is invisible in the parameters themselves. A scale term of −20.489 parts per million and one of −19.045 look equally plausible, and nothing in either says that one of them has a strain field folded into it. The only way to know is the residual — which is why the seven parameters insists that a parameter set published without an RMS is a claim without an error bar.

What the residual costs in practice #

A metre and a half of RMS is easy to read as small, next to translations of five hundred, and the comparison is the wrong one. What matters is the tolerance of the work the coordinate is going into.

A boundary survey works to a few centimetres. A construction set-out works to millimetres over a site. A national mapping product at 1:1250 has a specified accuracy of about half a metre. Every one of those is smaller than the residual a seven-parameter fit leaves, which means the seven parameters are not a conversion for those purposes; they are an approximate conversion with an unstated error larger than the job.

That is why the published transformations come with a stated accuracy, and why the good ones come with a table instead. Britain’s national transformation is a grid at one kilometre; North America’s successive versions are grids; Australia’s are grids. In each case the seven parameters still exist and are still published — as the approximate route, explicitly labelled, for the applications where a metre does not matter.

Why more parameters is the wrong instinct #

The natural response to a residual with structure is to model the structure, and it is worth following that road far enough to see where it ends.

Adding quadratic terms to the transformation would absorb the field used here exactly, since the field is quadratic. It would leave the cubic part of a real distortion, which is smaller and is not zero. Adding cubic terms leaves the quartic. Each round of additions buys less than the last, and each additional parameter is estimated from the same finite set of markers, so its own uncertainty grows as the parameters compete for the same information.

There is a name for the end of that process. When the number of parameters reaches the number of markers, the fit is exact and the model has become a lookup table with interpolation — which is precisely what a grid-shift file is, and what when a formula is not enough puts a spacing on. The mapping agencies did not fail to think of higher-order models. They followed the argument to its conclusion.

The interesting consequence is that the seven-parameter form survives not because it is accurate but because it is portable. It is seven numbers in an email; a grid is a file with a format, a version and a licence.

Where the model stops #

The strain field here is synthetic and says so. Its amplitude and its shape are stated inputs. What is being demonstrated is that seven parameters have seven degrees of freedom and a strain field has more, which is a statement about arithmetic and needs no real markers to be true. The magnitude — a few parts per million — is chosen to match the network distortions that published grid-shift files actually carry, and it is the same order as the spread a locally fitted figure leaves in a datum is fitted to a region, which is not a coincidence: both are what a low-dimensional surface cannot follow.

Real network distortion is rougher than this. A quadratic is the smoothest thing a similarity transformation cannot reach, so it is the kindest case. Actual triangulation error has structure at the scale of individual chains, which is shorter, and that is exactly what makes the tables in when a formula is not enough need the spacing they need.

More parameters do not fix it. The obvious response is to fit ten parameters, or twenty, with quadratic terms included. That works, and it moves the problem rather than solving it: the strain then has cubic structure the quadratic model cannot reach, and each additional parameter is fitted from the same finite set of markers and therefore has its own uncertainty. The end of that road is a model with as many parameters as markers, which is a table — and a table is what the mapping agencies ship.

The generalisation #

The shape of this argument recurs wherever a low-dimensional model is fitted to something that is not low-dimensional, and it has three parts that are worth separating.

A model’s residual is a measurement of the model, not of the data. The 1.64 metres is not an error in the markers. It is the part of the truth that seven parameters cannot express, and it would be exactly the same if the markers were perfect and the strain were real — which, in the actual case, it is.

A residual with structure is a different animal from a residual without. Noise shrinks with more observations; structure does not. The cheap test is neighbours-against-shuffled, and it needs no theory about what the structure should look like.

The fitted parameters are contaminated by whatever the model cannot express, in proportion to how badly the sample breaks the symmetry that would have kept them orthogonal. This is the least intuitive part and the most portable: parameters from a fit are not properties of the system, they are properties of the fit, and comparing two published parameter sets is comparing two samples as much as two systems.

The same pattern is behind a finding two rungs down this collection. A score computed over a different sample is not the same score — an aspect sweep that scored each candidate over whatever ground it could show announced that the best way to map an equatorial band is a transverse aspect, because a third of the tropics had silently left the average. Different subject, same failure: the estimate absorbed a property of the sample.

A larger strain, and the same shape #

That linearity is worth confirming rather than assuming, because it separates two things a single figure cannot: how much of the network’s error a fit leaves, and which part. The first scales; the second does not change at all.

The same field, resampled #

A residual field is a picture as much as a table, and moving that picture into another projection is an operation with a measurable cost — one the applied field measures on a synthetic field for exactly the reason this essay uses a modelled strain rather than a real network.

Warping a grid into another projection and back moves no coordinate: the maps are exact both ways. What is lost is that a target cell’s centre does not fall on a source cell’s centre, so a value has to be invented for it. Measured by refining the grid, the invention converges at first, second or third order depending on the kernel — and a second round trip loses more than the first, because a filter applied twice is a narrower filter.

The fitted orders are what turn a preference between kernels into a statement, and they are also why one claim that first accompanied this measurement had to be withdrawn: nearest-neighbour is not a filter, so its change in variance has no sign, and a sentence about contrast that held for the two smoothing kernels did not hold for it.

Who found it, and when #

The people who first saw this were the ones doing the adjustments, and they saw it in the 1950s and 1960s as soon as there were two independent realisations of anything to compare.

The mathematics was not new — least squares and its residuals go back to Gauss, who invented both in the course of exactly this kind of work — and neither was the observation that a triangulation accumulates systematic error along its chains. What was new was having a second, independent measurement of the same country, good enough that the difference between the two was the older survey’s error rather than the newer one’s.

Satellite geodesy supplied that in the 1970s — the same development that made the placement of a national ellipsoid measurable for the first time, as what a coordinate refers to records — and the answer everywhere was the same: the difference between an old national network and a space-based frame is a rigid motion plus a distortion field of a few parts per million, and the distortion is the older network’s. Britain’s OSTN transformation, North America’s NADCON and its successors, and Australia’s grids are all responses to the same finding — that after the best seven parameters have been applied, metres remain, in a pattern.

The naming is unfortunate in the usual way. These are called transformation grids or distortion grids, which suggests a correction to a transformation. They are better understood as the definition: the relationship between two realisations is a table, and the seven parameters are a compression of it that happens to capture most of the variance.

The relationship between the two figures is the honest summary of the whole field. The datum shift is hundreds of metres and is corrected. The residual left by correcting it is a metre or two and is not. And a metre or two is above the tolerance of a boundary survey, a construction set-out and a large-scale mapping product — so the part that gets attention is the part that has been dealt with, and the part that is left over is the part that decides whether the answer is usable.

Reading a published fit #

Four questions, and a published transformation that answers none of them is a number without an error bar.

What was the residual? Not the parameters’ formal uncertainty — the RMS of the fit at the stations. That is the part of the answer the model could not express, and it is the number that decides whether the transformation is usable for a given job.

Over which stations? As the wedge figure shows, the parameters are a property of the sample as much as of the two datums. A set fitted over a whole country and a set fitted over one county will differ, and neither is wrong.

Over what region is it claimed? A similarity transformation is defined everywhere and is fitted somewhere. Applied outside its fit region it produces plausible numbers with no error bar, which is the failure mode distortion over a region records in the projection half of this subject: a quantity computed over one region and quoted over another has changed meaning, not merely accuracy.

Is there a table instead? If the agency publishes a grid, the parameters are the fallback and the grid is the definition. Using the parameters when the grid exists is choosing a metre of error for the convenience of seven numbers, which is sometimes right and should be a decision.

Where this goes next #

If seven parameters cannot carry the relationship, something has to. When a formula is not enough works out what a table costs — what grid spacing a stated tolerance buys — and finds that the answer is set by the roughness of the difference rather than by its size, which is why a shift of a hundred metres can be tabulated coarsely and a ripple of thirty centimetres cannot.