A cocked hat holds the ship one time in four

A line of position is Newton’s method, but only on a conformal chart measured what the intercept construction adds to perfect sights: on Mercator, an error that falls as the square of the assumed position’s, which is nothing a navigator has to worry about. It ended where navigation actually starts. Sights are not perfect, a navigator takes three rather than two, and three lines of position almost never meet at a point.

They bound a small triangle, which navigators have called the cocked hat for as long as there have been three lines to draw. The universal instinct is that the ship is somewhere inside it and that a smaller hat is a better fix. The first is true a quarter of the time. The second depends on what kind of error made the hat.

A quarter, whatever the shape #

Every sight here is taken from a ship whose position is known, with a random error added to each altitude: a standard deviation of two kilometres along the body’s azimuth, a little over a minute of arc, which is an ordinary error for a sextant at sea. The three lines are reduced from the ship’s own position, so the construction adds nothing and the triangle is made entirely by the sights.

Twenty thousand hats for each of five arrangements of the three bodies — spread round the ship, evenly all round it, bunched on one side, bunched narrowly on one side — contain the ship 25.2 to 25.3 per cent of the time. The standard error is 0.3 per cent, and every one of the five is within it of a quarter.

The reason is a counting argument, and it is short enough to carry in the head. Each line of position misses the ship on one side or the other. With an error that is as likely to be short as long, each side is equally likely, and the three lines’ sides are independent, so the eight arrangements of three sides are equally likely. The triangle contains the ship in exactly two of them: when every line passes the ship on the same rotational side, either all clockwise or all anticlockwise. Two in eight is a quarter.

Nothing in the argument uses the triangle’s shape, the angles between the lines, the size of the errors or even that they are Gaussian — only that each is as likely to fall on one side as the other and that they are independent. That is why the five arrangements agree to the width of their own sampling error, and why a hat that is small, or nearly equilateral, is not a hat more likely to contain the ship.

What the triangle’s size does measure is the sights’ agreement with each other. The answer is a set made the same point about identifying a projection: a best fit without a spread is not a measurement. The cocked hat is a spread of a kind, and it is the wrong kind to read as a region the ship is in: it is a region the three sights disagree about, and the ship is outside it three times in four.

What the size of a hat says, and what it does not #

The most common reading of a cocked hat is that its size is the quality of the fix: a small triangle means sights that agree, and sights that agree put the ship close to where they meet. The first half is true and the second does not follow from it.

With independent errors, the hat’s size is a clean measure of the sights’ precision. The same random draws scaled from half a kilometre to five give a mean area that grows exactly as the square of the error — 1.776 times the square of the standard deviation, at every size — and a share of hats containing the ship that does not move at all: 25.27 per cent at every scale, because nothing in the counting argument depends on how large the errors are.

But for a given set of sights, the size of this particular hat says almost nothing about how far this particular fix is from the ship. Sorting twenty thousand hats by area, the smallest third average 0.44 square kilometres and the largest 17.45, forty times larger. The centre of a hat in the smallest third is 2.04 kilometres from the ship on average, and the centre of one in the largest third 2.09. The small hats are not better fixes.

They are, however, much worse at containing the ship. A hat in the smallest third contains it 2.5 per cent of the time; one in the largest third, 54 per cent. That is not a paradox. A small hat is three lines that happen to pass close to one point, and the point they pass close to is the sights’ consensus rather than the ship, which is the same two kilometres away as ever; a triangle that has shrunk to a dot has shrunk away from the ship, not onto it. How big a triangle it takes reads a surveyed triangle’s closing error against the instrument’s noise in the same spirit: a small misclosure on one triangle is luck, and only the spread over many says what the instrument is worth.

A common error hides differently in the two skies #

A common error also changes the hat’s size, and not by the same amount in the two skies. With no independent error at all, a common error of three kilometres makes a hat of 47.7 square kilometres when the bodies surround the ship and one of 5.2 square kilometres when they are on one side. With the two kilometres of independent error added, the mean areas are 54.7 and 12.4.

So when the bodies surround the ship, a common error announces itself: the hat grows several times over and the ship sits in its middle. When they are on one side, the same error barely enlarges the hat and moves the ship out of it. A navigator on one side of the sky sees a hat that looks like ordinary sights, and it is wrong in exactly the way a hat can be wrong without looking it. A crossing is a chain of decisions found that a forecast wrong by a steady bias and a forecast wrong by noise call for different corrections; this is the same distinction, arriving at a fix instead of a route, and with the geometry of the sky deciding which kind of wrong is visible.

That is the reason behind a piece of standing advice. Navigators are told to choose three stars spread round the horizon, about a third of the compass apart, rather than three bunched together. The measurements say what the advice buys: not a smaller hat, but a hat in which a common error, if there is one, puts the ship inside and makes the triangle larger rather than hiding it. Where the control points are makes the same point about a fit: which parameters the observations can recover is decided by where they were taken, before any of them is measured, and the sheet moved before it was measured is about a systematic error that a badly placed set of observations absorbs without trace.

A common error changes the question #

Not every error is independent. A sextant with an index error misreads every altitude by the same amount; so does a wrong allowance for the height of the observer’s eye. Every line of position then moves the same distance along its own azimuth — towards every body, or away from every body — and the three lines are no longer scattered at random about the ship.

With the bodies spread all round the ship, adding a common error of three kilometres to the independent two raises the share of hats containing the ship from a quarter to 82 per cent, and six kilometres raises it to 99.6. With the bodies bunched on one side, the same common error lowers it to 6.8 per cent at three kilometres and 0.1 at six.

The same sextant, misreading by the same amount on the same night, makes the triangle almost certainly right in one sky and almost certainly wrong in another.

Decided at a half-turn #

What separates the two skies is one geometric fact about three lines moved the same distance.

If every line has moved the same distance, the ship is the point the same distance from all three. For any triangle there are four such points: the centre of the circle that fits inside the triangle, and the centres of three circles that touch each side from outside. Which of them the ship is depends on which way each line moved — and a common error moves every line either towards its body or away from it.

When the three bodies surround the ship, spanning more than a half-turn of the compass, moving every line towards its body moves every line inwards, or every line outwards, and the ship is the inner centre: inside the hat. When the bodies are all on one side, spanning less than a half-turn, the lines that moved towards their bodies no longer close round the ship: it ends up beyond one of them, on the far side from the other two, and it is one of the outer centres — outside the hat.

The figure sweeps the three bodies’ span from 90 to 240 degrees. Without a common error the share is a quarter throughout, as the counting argument says it must be. With one it is a step at 180 degrees: below the half-turn at most 6.8 per cent, above it at least 81.5.

Solving for the common error #

A navigator who suspects a common error can solve for it. Three lines of position, each with an unknown shared shift along its azimuth, are three linear equations in three unknowns — the two coordinates of the ship and the shift — and they have one solution: the point equidistant from all three lines. It is to the cocked hat what a residual has more than one explanation is to a fit: a reading of the misfit as a quantity to be estimated, rather than as noise to be averaged.

With the bodies all round the ship, the equidistant point and the centre of the hat are almost the same point, and both are about 2.05 kilometres from the ship on average at any common error. The choice hardly matters.

With the bodies on one side it matters a great deal, and in both directions. The centre of the hat is 2.03 kilometres out with no common error and 8.13 kilometres out with six: it carries the common error in full, because a common error on one side of the sky moves the whole triangle away from the ship. The equidistant point is 4.30 kilometres out at every common error, because it has solved for the error and removed it — but twice as far out as the centre when there was nothing to remove. Three lines whose azimuths all lie in one half of the compass pin a shared shift badly, and solving for it spends precision that the independent errors then fill.

The two cross near three kilometres of common error. Below that, trusting the independent errors and taking the centre of the hat is better; above it, solving for the shift is. That is the ordinary price of an extra unknown: estimated from the same observations it costs precision, and it is worth paying only when the thing it absorbs is larger than the noise. The datum hides inside the projection’s parameters meets the other side of it, a fit that absorbs a shift nobody asked it to estimate; here the shift is asked for, and the scatter is the bill.

The hat on a sheet that forgets the latitude #

The last measurement brings the chart back. A navigator plotting on a sheet whose longitude is not corrected for latitude gets a first-order error in every line of position, and the essay before this one found it to be one less the cosine of the latitude, of the east–west part of the assumed position’s error.

On Mercator the share of hats containing the ship stays at a quarter whatever the assumed position’s error — 25.4 per cent at twenty kilometres. On the uncorrected sheet at 50° north it is 25.1 per cent at one kilometre of assumed-position error, 13.6 at five and none at twenty.

And the hats are the same size: 7.01 square kilometres on average on both sheets. The first-order error does not make the three lines disagree more; it moves all three together, so the triangle keeps exactly the shape the sights gave it and is drawn several kilometres from where it belongs.

That is the most dangerous combination a plot can produce. A navigator reads a small, well-formed hat as a good fix, and on this sheet a small, well-formed hat is exactly what a wrong geometry draws. Nothing about the triangle says it has moved. The plate carrée, the projection nobody chooses is about the map that happens when nobody decides on one; a plotting sheet laid down without its latitude correction is that map at the scale of a single fix, and at 50° north, from an assumed position twenty kilometres out, it takes the ship out of the hat every time.

What each number was checked against #

The ship is known. Every sight is computed from the ship’s true position and then given its error, so containment is a test against a fixed point, and every share is a count.

Perfect sights from the ship make no hat. With no error and the ship as the assumed position, all three lines pass through one point and the triangle has no area to the arithmetic. A construction that left a hat there would be making its own.

A quarter must be a quarter. For a layout that surrounds the ship and one that does not, the share with independent errors must lie within three standard errors of a quarter, and does, at 20,000 hats each.

A common error alone must decide by the half-turn. With no independent error, a common error of three kilometres must put the ship inside the hat for bodies all round it and outside for bodies on one side, and does.

And the uncorrected sheet must keep the hat’s size and lose the ship: at twenty kilometres of assumed-position error, a mean area within two per cent of Mercator’s and a share below one in twenty. The area agrees to the three figures printed and the share is nought.

What three bodies and a known ship leave out #

The errors are stated, not observed. Two kilometres independent and a few kilometres common are ordinary magnitudes for a sextant, and real errors have correlations the model leaves out: two bodies observed low in the same haze share part of their refraction error, which is neither independent nor common.

The three bodies are observed at once. Sights taken minutes apart from a moving ship must be run up to a common time, and an error in the ship’s run is another shift of the lines — along the ship’s track rather than along each azimuth — that is neither kind measured here.

The triangle is judged by containment. A navigator who treats the hat as a region to be in is the navigator this essay is about. One who treats its size as an estimate of the sights’ precision, and its centre or its equidistant point as the fix, is using it as the measurements above say it can be used.

Still open: more lines than unknowns #

Four sights give four lines and no triangle, and a navigator with four has to choose a point by some rule. The obvious rule is least squares: the point that minimises the sum of the squared distances to all four lines, which is what the answer is a set would call a best fit, and what a satellite receiver does with its ranges.

With independent errors that is the right point. With a common error it inherits the bias in full, as the centre of the hat does, unless the common error is solved for as a fifth unknown; and with four lines the choice between those is no longer forced by the count, because four equations can estimate three unknowns and still leave a residual to judge them by. How often the least-squares point of four sights lands within a stated distance of the ship, whether a residual can reveal a common error that the fix itself hides, and how much solving for it costs when the bodies bunch on one side, are questions three lines cannot ask.