Cartographic Projection
every map is wrong, and by exactly how much
Concept

Standard error — where it appears

The formal uncertainty an adjustment reports for a coordinate, computed from the observations' assumed precision and the network's geometry. It describes what happens if every observation is merely noisy, and says nothing about what happens if one of them is wrong.

Named by 10 essays across 5 fields — each of them below, with the objects they name alongside it.

Bigger triangles, and how much bigger depends on what is fixed. How well a survey can resolve Gaussian curvature, against the side of its triangles, under three things being held fixed. One triangle: the accuracy improves as the inverse SQUARE of the side, fitted exponent -2.000. A chain of fixed length, which is what every great arc was: bigger triangles mean fewer of them, and the exponent is -1.503 — exactly three halves. A network covering a fixed area: -0.999, exactly one. The trade depends on what a survey is short of.

How many triangles it takes

Pricing one triangle left the fact that a survey observes hundreds unaccounted for. Averaging n of them divides the noise by √n, and n is not free: a chain of fixed length holds fewer big triangles than small ones, so the accuracy improves as the side to the power three halves rather than two. Struve's 141 triangles of forty kilometres are worth exactly Gauss's seven of eighty-five.

impossibility · Curvature How large a blunder has to be before the test notices. A blunder of increasing size put into the least-checked observation of a braced quadrilateral, with the standardised residual it produces. The horizontal line is the critical value the test uses, and the vertical one is the minimal detectable bias — δ₀σ/√r, which is 87 millimetres for this observation and is computed from the network's DESIGN, before any observation is made. Below it nothing is flagged; above it everything is. The observation's redundancy number is 0.144, so it is checked by a seventh of an observation and hides six-sevenths of whatever is wrong with it.

The blunder the network cannot see

A least-squares adjustment has no concept of a mistake. The smallest blunder its test will find in the least-checked leg of a braced quadrilateral is 87 millimetres, and by the time it fires a station has moved by nearly ten times the accuracy the same adjustment reports for it.

practice · Reduction A transformation is more certain in some places than others. The horizontal position uncertainty a Helmert transformation carries, from the stated widths of its own seven parameters, along three meridians. It runs from 40 mm to 68 mm — a factor of 1.71 — and it falls towards the poles, because the rotation terms act on the distance from the Earth's axis. A single figure quoted for "the accuracy of the transformation" is the value at some latitude nobody wrote down.

The seven parameters have their own uncertainty

Nine essays on this ladder print a datum transformation as seven exact numbers. Every published set is the output of a least-squares fit and arrives with standard errors as much a part of the result as the parameters — and pushing those widths through to the ground gives an ellipse, not a number, that is 68 mm across at the equator and 43 mm at 70°.

datums · Datum Weighting the low sight turns the region the fix is in. Four bodies, three of them between 44° and 58° up on one side of the ship and one 5° up behind, with each sight's error made of a sextant part of 0.5′ and a tenth of its refraction correction: 0.94, 0.95, 0.93 km for the high bodies and 2.05 km for the low one. Solid: the 95 per cent ellipse of the fit that weights each sight by its variance, 2.30 by 1.55 km. Dashed: the ellipse a navigator draws who gives every sight the same variance, 2.63 by 2.02 km, its long axis turned 20.1° from the weighted one. Dotted: where that unweighted fix actually scatters, 2.98 by 1.57 km, turned 27.7° from the ellipse drawn for it. All three are centred on one point.

A low sight is worth keeping only if it is weighted

A body five degrees above the horizon is seen through so much air that its refraction correction is ten times a high body's, and a tenth of that correction uncertain doubles the sight's error. Given the same weight as the others, a sight below seven degrees makes the fix worse than throwing it away; weighted by its variance it never does. The ellipse drawn without weights is turned twenty degrees from the right one, and a navigator who must guess the weights loses far less by guessing high.

paths · Position fix Weighted by their residuals, six sights hand the low one six times its share. Six bodies seen from one ship: five between 44° and 61° up between east-north-east and east-south-east, and one 5° up due south, alone in its direction. Along each bearing, two bars: the share of the fix's weight the sight's altitude earns it, and the median share it ends with when every sight is weighted by its own residual and the fit is repeated until the weights settle. The low sight earns 4.0% and ends with 25.0%; the five good ones earn about 19% each and end with 1.8% to 3.1%. Bar length goes as the square root of the share.

Weighted by what it disagrees with, the fix trusts the sight it leans on

With six sights and two unknowns, the residuals seem to offer a way to find the bad sight without knowing the weather: weight each line by how little it disagrees, and solve again. One step of that gives a low body almost exactly the weight its altitude would, 4.5 per cent against 4.0. Every further step walks it away. Iterated until the weights settle, the fit is worse than weighting nothing in all four arrangements tried. Where the low body stands alone in its direction it ends with six times the weight it earns, because the fix leans on its line and its residual shows only 46 per cent of its error.

paths · Position fix A week of twilights learns the refraction share, and one twilight learns almost nothing. The share of the refraction correction that is uncertain, estimated from the residuals of equal-weight fixes pooled over one to fifty-six twilights, each residual corrected exactly for its own fix's geometry; the truth is a tenth. Shaded: the middle eighty per cent of 1,500 seeded weeks for a navigator who always takes one body between 3° and 8°, with its median solid. Dashed: the median for one who takes six bodies anywhere between 4° and 70°. After one twilight the first navigator's middle eighty per cent runs from 0.000 to 0.181; after fourteen, from 0.055 to 0.131; after fifty-six, 0.079 to 0.116. The medians climb to the truth from below: 0.049 and 0.020 after one twilight, 0.092 and 0.088 after fourteen.

A week of twilights learns what a sight is worth

One fix cannot tell how good its low sight was, but every residual of every fix is a point on one straight line whose slope is the square of the refraction share, once each is corrected exactly for its own fix's geometry. After one twilight the estimate is zero four times in ten; after a week it is within half again of the truth nearly four times in five. And the lesson is worth little: a guess within a factor of two already costs under 4 per cent of the fix, a week's estimate 1.3, and a share that changes from night to night by half again costs more than the learning buys.

paths · Position fix Tonight's residuals cost the fix a steady three per cent; the typical share costs more every time the nights differ more. Six bodies a twilight, 3 of them between 3° and 8°, and tonight's refraction share the typical tenth times a factor drawn afresh each night; the horizontal axis is that factor at one standard deviation. The fix error, root-mean-square over 40,000 seeded twilights, as a percentage above that of a navigator who knows tonight's share. Dashed: weighting with the typical share, 0.8 per cent at a factor of one and a quarter, 10.3 per cent at two and 36.2 per cent at three, off the top of the chart. Solid: weighting with tonight's share estimated from tonight's own residuals, 3.6 per cent, 3.2 per cent and 1.9 per cent. Dotted: an even blend of the two, 1.2 per cent, 2.3 per cent and 2.0 per cent. Tonight's estimate beats the typical share from a factor of about 1.56, and the blend from about 1.33.

Tonight's residuals measure tonight's air badly and weight tonight's fix well

A week of twilights learns the typical refraction share and nothing about tonight's, and a share that changes from night to night costs more than the week buys. With the sextant's error taken from the week, one twilight's residuals estimate tonight's share alone — badly: within half again of the truth 43 times in a hundred with three bodies near the horizon, and zero 22 times. Used as a weight, that poor estimate costs the fix between two and four per cent whatever the night, so it beats the typical share once nights differ by more than half again, and a blend of the two, leaning on tonight as the nights grow more variable, beats both. A weather predictor has to explain most of the nightly change to do better.

paths · Position fix Waiting longer buys almost nothing under the spectrum that is actually there. The scatter of block means against the block length, for the three spectra. White noise falls with a slope of -0.484 — the inverse square root everybody assumes. Flicker falls with a slope of -0.111, so a hundredfold longer average buys a factor of 1.67 rather than ten. A random walk is very nearly flat.

The error that does not average down

Five rungs give a coordinate a width, and all five assume the observations it was averaged from are independent. They are not. Under the noise spectrum every published analysis of a position time series reports, the standard error falls as the eleventh root rather than the square root — so a factor of ten costs a hundred observations if the noise is white and a thousand million if it is not.

distortion · Precision The bias is σ² over the length, over two decades. The amount by which a measured baseline is longer than the true one, against its length, for a twenty-millimetre error on each end. The line is the second-order prediction σ²/d. The measured bias times the length is constant to a factor of 1.0014 across the whole range, and it sits 8.2 per cent below the prediction — which is the fourth-order term the expansion drops. The estimate is antithetic, so the first-order scatter cancels exactly and a bias of thirty-seven microns is measured at a t-statistic of 349.

A length measured from noisy points is too long

A distance is a square root, a square root is concave, and the average of the distances is not the distance between the averages. The gap is a bias with one sign: 37 microns on a ten-metre baseline with twenty-millimetre marks, following σ² over the length across two decades, and it adds rather than cancelling — so the same boundary is 1.5 parts per million longer when it is measured in more pieces.

distortion · Precision Two places, one written coordinate. The lattice a coordinate written to 5 decimal places lives on, at 45° of latitude, where a cell is 1.11 metres north to south. Two real places a fifth of a cell apart are drawn as open marks and the single value they both round to as a filled one. The distance between them, computed from what was written, is exactly zero — an error a noise model cannot produce, since independent noise never puts two different points in the same place.

Rounding is not noise

Eight rungs treat a coordinate's error as noise that averages down. A published coordinate has a second error that does not: it is deterministic, it is shared between every point in the same cell, and at five centimetres apart ninety-three per cent of pairs come out as the same place — which no amount of independent noise can produce.

distortion · Precision

Named alongside it

The objects these essays reach for when they reach for this one.

Least-squaresNoiseVerificationCovarianceError budgetEstimatorNavigationPrecisionResidualWeightingAveragingSurvey

All concepts