The weights are a guess the solve believes

A coordinate is the output of a solve finds a decision buried in a least-squares adjustment that no residual can see. The problem has a datum in it — which station is held, which bearing is fixed — and it is not a measurement. Change it and every coordinate moves by centimetres while not one residual moves at all, to 5 × 10⁻¹³ metres.

There is a second such decision. It is made far more often, it is thought about far less, and it does something the datum does not: it changes the answer in a way that is genuinely better or worse rather than merely different.

Every observation enters the adjustment with a weight, the weight is one over the square of a stated standard deviation, and the stated standard deviations come from a manufacturer’s specification, a rule of thumb, or the last job. A network mixing two kinds of observation has a ratio between them, and nobody measured it.

This is not a small corner of the practice. A published coordinate is a result establishes that the number on a certificate is the output of an adjustment rather than of a measurement; the tolerance decides the model establishes that what a job is for decides how much of the geometry it has to carry. Between them they say that a coordinate is a computed thing with stated inputs — and the weights are the one input that is neither measured nor stated.

The ratio moves the coordinates #

Two things in that curve are worth separating.

The penalty for being roughly right is nothing. Anywhere between a ratio of one and sixteen the coordinate error sits between 25 and 28 millimetres. A surveyor who guesses the ratio within a factor of four has lost nothing measurable, which is why the practice survives.

The penalty for being wrong in the wrong direction is real. Believing the worse instrument is what costs, and it costs 76 per cent. That is the asymmetry: over-weighting an accurate group is nearly free, and over-weighting a noisy one is not.

The check everybody runs cannot see it #

Every adjustment reports a variance of unit weight, σ₀ — the weighted residual sum of squares over the degrees of freedom — and every textbook says to check that it comes out near one. If it is much larger the observations disagree with their stated precisions; if much smaller the precisions were pessimistic.

That check cannot see a ratio, and the reason is arithmetic rather than statistical.

A test that can always be passed without changing the answer is not a test of the answer. σ₀ is one number and it responds to the overall level of the stated sigmas; the ratio between two groups is a second number, and one number cannot carry two. An adjustment can be brought to σ₀ = 1 at any ratio whatever, which is precisely what a practitioner does when it comes out wrong.

What would test it, and what happens to it #

The test that can see a ratio is Helmert’s variance components. Split the weighted residual sum of squares between the groups, divide each by that group’s own share of the redundancy, and each quotient is an estimate of that group’s variance. It is unbiased, it is standard, and it has been in the surveying literature since 1924.

And here is the thing this rung exists for.

Up-weighting a group takes away the redundancy that would have tested it. The mechanism is direct: a heavily weighted observation has its residual driven towards zero, a residual of zero contains no information about that observation’s error, and a group’s own variance can be estimated only from its own share of the redundancy. At the true ratio of four, the accurate edges carry 0.99 degrees of freedom out of thirteen — twenty-four observations, one degree of freedom between them.

So a surveyor confident about an instrument has disabled the arithmetic that would have caught the confidence. That is not a paradox and it is not anybody’s fault; it is what “believing an observation” means, written out.

And on this network the iteration does not save it #

Helmert’s method is iterative because the first estimate is biased by the weights it was computed under. As the weights improve, the bias falls, and on a well-conditioned problem the iteration walks to the truth.

On this network, started from equal weights, it does not. The two groups’ true errors are 3.56 and 16.64 millimetres, a factor of 4.7 apart. Their residuals are 6.69 and 7.43 millimetres, a factor of 1.11 apart — because a badly-weighted adjustment spreads one group’s error across the other’s residuals, and the first estimate therefore sees two nearly identical numbers. The iteration has nothing to walk on and drifts to a ratio of 0.77 against a true 4.

That is a fact about this geometry rather than about the method, and it is reported rather than hidden. The estimator is unbiased at the truth and does not find the truth from a distance, and both halves are measured here.

The same shape as the datum, one level down #

It is worth putting the two decisions side by side, because they look alike and behave differently in a way that decides what to do about each.

The datum is invisible and harmless. Change what is held and every coordinate moves; change it back and they move back; and the residuals do not move at all, to 5 × 10⁻¹³ metres. Nothing is better or worse — the two answers are the same network in two frames, and the ladder’s seventh rung shows that the choice is a statement about what the coordinates are for rather than about what they are.

The weights are half-visible and not harmless. Change the ratio and the coordinates move, and one of the answers is closer to the truth than the other. The residuals do move, so the decision leaves a trace — and the trace is weakest exactly where the decision is most confident.

So the two need opposite treatments. The datum should be stated, because it cannot be discovered and a reader has to know which one was used. The weights should be tested, because they can be, and the test is a property of the design rather than of the observations.

What is actually available before going out #

The useful part of all this is that none of it needs an observation. The redundancy split is a property of the network’s geometry and the weights it will be adjusted under — it is computable from the design alone, before anybody sets up an instrument.

So a network can be designed so that each group carries enough redundancy for its own weight to be testable, in the same way it is designed so that each observation carries enough for a blunder in it to be visible. What a closed figure cannot see makes the second argument for a traverse and finds a class of error the closure check is blind to; this is the same argument one level up, about the weights rather than about the observations.

And the design lever is the obvious one. The accurate group loses its redundancy because twenty-four edges are nearly enough to determine twenty-nine unknowns on their own, so they can be fitted almost exactly. Adding lines within that group — a second diagonal in each cell, a longer edge — puts redundancy back where the test needs it, at a cost that is a day’s work rather than an instrument.

Where the weights go after the solve #

The weights do not stop mattering when the coordinates come out. They are carried forward into everything the adjustment reports about its own quality, and that is where a bad ratio does its second kind of damage.

That matters because the ellipse is what gets used. An error ellipse is an indicatrix shows that it is the same matrix Tissot’s is, pushed through the projection, so a wrong weight ratio propagates into every statement about a coordinate’s precision on the map. The difference of two coordinates shows the same for a baseline, where the relative ellipse is six times smaller than the absolute ones and depends on the same weights through the same cofactor block.

And it matters for the operation everybody does next. The two ways to spread a misclosure prices the difference between distributing an error by distance and by latitude and departure — both of which are weighting rules, chosen for tradition and convenience rather than measured, and both of which are the same decision as this rung’s under another name.

That last figure is the point of the whole rung stated at its sharpest. Believing a group of observations makes their weight untestable and their blunders invisible at the same time, by the same arithmetic, and the number that says how much is available before anybody leaves the office.

What a practitioner should take from this #

Four statements, in the order they are worth acting on.

Guess the ratio in the right direction. The curve is asymmetric: over-weighting the accurate group costs almost nothing and over-weighting the noisy one costs the most. Given a choice between two plausible ratios, take the one that trusts the better instrument more.

Do not read σ₀ as a check on the weights. It is a check on their overall level and on nothing else, and it is passed by a rescaling that changes no answer. Reporting it is fine; treating it as evidence that the ratio is right is not.

Compute the redundancy split before the job. It needs the design and the intended weights, no observations, and one matrix inversion. It says which group’s weight the job will be able to test and which it will not, at a point where the design can still be changed.

And add lines within the group being believed. That is the only lever that puts the test back. The accurate group here loses its redundancy because twenty-four edges nearly determine twenty-nine unknowns on their own, so they can be fitted almost exactly; extra observations among those same stations are what makes them checkable.

None of this is expensive and none of it is new mathematics. What is new is putting the redundancy split and the weight ratio on one axis, and it is one figure.

Where the model stops #

The two groups are stated and their true sigmas are known. That is the only way to score anything: the experiment measures whether a method recovers a quantity, so the quantity has to be put in. No real job knows its own sigmas, which is the entire reason the question exists.

The noise is independent between observations. Real observations of the same network share errors — a mis-centred instrument affects every line from that station, a temperature error affects a whole afternoon’s work — and correlated errors break the variance-component estimator in a way this measurement cannot see.

Only distances are observed. A real network mixes distances with directions, and that is the classic setting for variance components because the two have different units and no natural ratio at all. The distance-only case here is the harder one to demonstrate on and the easier one to reason about; the mixed case has the same structure with a units conversion in front of it.

And the coordinate error is measured against a truth. A real adjustment has no truth to compare against, which is why σ₀ and the variance components are what is available. The curve in the second figure is the thing nobody can see, drawn so that the things everybody can see have something to be compared with.

Who found it, and when #

Friedrich Helmert set out the variance-component estimator in 1924, in a paper about combining observations of unequal precision, and it has been standard in the geodetic literature ever since — with a substantial body of work on when it converges, when it produces negative estimates, and how to stabilise it.

What is missing from that literature, as far as this collection can tell, is the redundancy figure above. The estimator’s dependence on the redundancy split is well known as a caveat; the fact that the split is driven to zero by the very weighting the estimator exists to check appears to be stated here for the first time, and it is one figure and no new mathematics.

The reason it is easy to miss is that the two facts live in different places. Redundancy numbers are taught as a blunder-detection tool — how visible is a gross error in this observation — and variance components are taught as a weighting tool, and both are computed from the same matrix and are almost never plotted against each other.

Two tools, one matrix, no shared page #

The reason the finding was available and unstated is worth its own paragraph, because it is the commonest way a fact goes missing in a mature field.

Both quantities come out of the same matrix. Redundancy numbers and variance components are computed from the design and the weights, in the same solve, by the same code. Nothing separates them technically and either could be printed beside the other at no cost.

They are taught as answers to different questions. A redundancy number answers would a gross error in this observation be visible, and it lives in the chapter on blunder detection. A variance component answers are these two groups of observations weighted correctly relative to each other, and it lives in the chapter on stochastic modelling. A student meets them weeks apart, in different contexts, with different worked examples.

So the relation between them has no natural place to be noticed. Seeing it requires plotting one against the other, which nobody has occasion to do because neither chapter’s questions call for it — and the relation is not subtle once plotted.

Which is the general shape. A field’s divisions are pedagogical and organisational, and they are usually good divisions, but they decide which pairs of facts ever appear on the same page. A relation between two quantities in the same chapter gets found in the first decade; a relation between quantities in different chapters can wait indefinitely, however elementary it is, because nothing brings them together.

A curriculum has to divide the material somehow, and any division has the same property.

And the remedy is not a better curriculum. It is the habit of computing a thing two ways and comparing, which is what this collection does everywhere and what turned up the relation here — not insight, but the discipline of putting two numbers from one solve side by side.

It is also the only remedy available to somebody who is not in a position to change how the subject is taught, which is everybody who meets the problem.

The general form is worth stating, because it is not about surveying. Every least-squares problem has a covariance matrix that somebody chose, and the estimate it produces is the best estimate given that choice — a conditional statement that the output does not carry. What the residuals test is whether the observations agree with each other under the assumed weighting, which is a weaker question than whether the weighting was right, and a network can pass it comfortably while being wrong about the relative precision of two instruments by a factor of three. The only thing that separates the two questions is redundancy of the right shape: observations that would disagree if the weights were wrong, which is a property of the network’s design and not of its adjustment. A design that cannot fail the test is a design whose weights were never in question and never confirmed either.

Where the ladder goes next #

The ladder has now found two decisions inside the adjustment that are not measurements: the datum, which changes the coordinates and no residual, and the weights, which change the coordinates and are testable only where the network was designed to test them.

There is a third and it is larger than both. The functional model — that a distance observation is a distance between two points on a plane, plus noise — is an assumption too, and it is the one that fails first in practice: an uncorrected refraction, a scale error in an instrument, a station that has moved since it was coordinated. Those are not blunders and they are not noise; they are the model being slightly wrong everywhere at once, which is the same shape as the network distortion the datum ladder finds on the other side of the site, and it is invisible to every check on this page.