Tonight's residuals measure tonight's air badly and weight tonight's fix well

A week of twilights learns what a sight is worth turned a navigator’s residuals into a regression. Every residual of every fix, corrected exactly for that fix’s geometry, is a point on one straight line whose intercept is the sextant’s variance and whose slope is the square of the refraction share — the fraction of the refraction correction that is uncertain, taken as a tenth in a low sight is worth keeping only if it is weighted. A week’s residuals put the share within half again of the truth nearly four times in five. One twilight’s put it at zero four times in ten.

The same essay then found the result that decides whether any of this matters. The fix is flat round the right share: a guess within a factor of two costs under four per cent of the fix, and a week’s estimate 1.3. But the refraction share is not a constant of nature. It is a property of the air near the sea surface, which changes with the weather, and a share that varies from night to night by half again costs more than the week’s learning buys — because a week teaches the typical night, and tonight is not typical.

What the navigator needs is tonight’s share, and the obvious source is tonight’s sights. The earlier essay named two routes to it: more than one low body in a twilight, so that tonight’s residuals carry more information about tonight’s air; and a predictor from the weather itself — the difference between the temperatures of the air and the sea, which drives the anomalous refraction near the horizon. This essay measures the first and prices the second.

One unknown instead of two #

The regression that failed at one twilight had two unknowns, the sextant’s variance and the square of the share, and six residuals with two already spent by the fix — four numbers for two unknowns, and the two unknowns nearly interchangeable. That is why a single twilight so often put the share at zero: a little more sextant error explains a little less refraction. And the residuals themselves are the part of each sight’s error the fix did not absorb — the residual reports the error the fix was immune to — so the sights the fix leans on hardest show least, which is how weighted by what it disagrees with came to trust its low body six times more than it earned.

The two unknowns are not alike. The sextant’s error is a property of the instrument and the observer, and it does not change with the weather. A week of pooling estimates it alongside the share, and it can be carried from week to week, so a navigator can take it as known and leave tonight’s residuals only one thing to estimate. With the sextant’s variance $\sigma_0^2$ fixed, each residual gives

$$r_i^2 - \sigma_0^2 \sum_j M_{ij}^2 \;\approx\; k^2 \sum_j M_{ij}^2 R_j^2,$$

a regression through the origin in tonight’s $k^2$, where $M$ is the equal-weight fit’s residual operator and $R_j$ is sight j’s refraction correction in kilometres. It is fitted by least squares with each residual weighted by its expected variance at the typical share, the estimate is floored at zero, and the fix is reweighted with it and solved again. Unlike the reweighting that walked away from the truth, this one step uses the residuals’ exact expectations, geometry included, and is not iterated. It is a cousin of what a fifth unknown costs when nothing can see it, which solved for a common error in every sight as one more unknown; here the unknown is a variance rather than an error, and a variance is harder to see.

Everything below is simulated rather than computed from a covariance formula, and that is a necessity rather than a choice. The weights now depend on the same errors the fix is made from — a sight that happened to be badly wrong raises tonight’s estimate and so lowers its own weight — and the sandwich formula the earlier essays used assumes weights fixed in advance. So each figure is 40,000 seeded twilights, six bodies each on random azimuths spanning at least a third of the compass. One to four of them are between 3° and 8°, the rest between 10° and 70°. Tonight’s share is the typical tenth multiplied by a factor drawn afresh each night from a lognormal distribution, and the spread of that factor — how far a night typically strays — is the axis that matters.

As a measurement, it is poor #

Taking the sextant’s error as known helps, and not by much. With one low body among six, tonight’s estimate lies within half again of tonight’s truth 29 times in a hundred and is zero 35 times. Three low bodies raise the first to 43 and lower the second to 22. The cloud of estimates follows the diagonal — nights with more refraction error do produce larger estimates — but at any true share the estimates spread over a factor of four or more.

That is about as well as four degrees of freedom can do. A variance estimated from a handful of residuals is always rough: its relative standard error from n independent squared normals is $\sqrt{2/n}$, which is 71 per cent at four and 58 at six, and here the residuals are neither independent nor equally informative, since only the low bodies’ residuals carry much of the refraction. By the standard a surveyor would apply to a measurement of anything, tonight’s residuals do not measure tonight’s air. The fix they come from is itself uncertain on the same scale — a cocked hat holds the ship one time in four is the reminder of how little a handful of lines pins down — and a variance is a second-order quantity estimated from what is left over.

As a weight, it is good #

The question a navigator cares about is not how close the estimate is to the air but how close the fix is to the ship. Measured that way the picture reverses. Weighting each twilight’s fix with its own estimate puts the fix 3.6 per cent further from the ship, root-mean-square, than weighting with tonight’s true share would, when nights stray by a factor of one and a quarter. At a factor of two it costs 3.2 per cent, and at three 1.9. The cost is nearly constant because it is the price of estimating, and the price of estimating does not depend on how far tonight is from typical.

The typical share’s cost does. It is nearly nothing when nights barely vary — 0.8 per cent at one and a quarter — because the typical share is then nearly tonight’s. At a factor of two it is 10.3 per cent and at three, 36. The two curves cross at a factor of about 1.56. Where nights differ from one another by more than half again, a navigator does better by trusting a poor estimate of tonight than a good estimate of the average night.

That is the earlier essay’s flatness doing the work. A week of twilights found that the fix’s error barely moves as the assumed share moves within a factor of two of the truth, because what matters is only whether a low sight is weighted roughly as a low sight. An estimate that is wrong by a factor of two is, for this purpose, nearly as good as the truth; the typical share, on a night far from typical, is wrong by more than that.

Why an estimate of zero costs so little #

The most alarming thing in the scatter is the row of zeros: a fifth of the twilights with three low bodies conclude that tonight’s refraction is perfectly known, and weight the low sights as though they were as good as the high ones. Sorting the twilights by how good their estimate was shows what those zeros cost. Twilights whose estimate came out zero lose 4.1 per cent of the fix. Those whose estimate was within half again of the truth lose 0.7. Those whose estimate was too high by more than half again lose most, 8.2.

The zeros are cheap for a reason the residuals supply themselves. A zero estimate is what a twilight produces when its low sights happen to agree with its high ones, and that is most likely on a night when the refraction error really is small. The median true share on the nights that estimated zero is 0.061, against 0.100 over all nights; 82 per cent of them are below the typical share. On those nights the typical share itself costs only 1.6 per cent, because the nights are near typical or below it, and treating the low sights as good is a smaller mistake than it sounds.

The costly nights are the ones where the estimate is too high: a low sight that happened to be badly wrong inflates tonight’s estimate and is then down-weighted more than it should be, which throws away some of the geometry the fix needed from it. Even so, those nights lose 8.2 per cent where the typical share would have lost 13.8 on the same twilights.

More low bodies raise the stakes, not the estimate #

The earlier essay asked whether three low bodies in one twilight could estimate the evening’s share. The answer from the scatter is that they estimate it better than one — 43 per cent within half again against 29 — and the answer from the cost is that this is not what changes. Tonight’s estimate costs 3.0 per cent of the fix with one low body, 3.3 with two, 3.2 with three and 2.1 with four. The typical share costs 5.3, 8.4, 10.3 and 9.5.

The more of a fix that rests on low sights, the more a wrong weight on them costs, and so the typical share becomes a worse choice as low bodies are added while tonight’s estimate stays about as good. A navigator with one low body loses 5.3 per cent to a variable night by using the typical share; with three, 10.3. At four low bodies the typical share’s cost falls slightly, to 9.5; the trials record that and do not separate its cause, though with four of six sights near the horizon the high sights, which the typical share weights correctly, are the minority. The second half of the answer is therefore the practical one: taking more low bodies is exactly the habit that makes tonight’s share worth estimating, and tonight’s residuals estimate it well enough to pay for itself at every count.

A blend beats both #

Neither end is the best use of what the navigator knows. The typical share is a good estimate of an average night and tonight’s residuals a rough estimate of this one, and the standard remedy for two estimates of different quality is to combine them. Taking the square of the share as a straight mixture — so much of tonight’s estimate, the rest the typical value — the best mixture beats both ends at every spread tried. Even an even blend, which ignores the spread entirely, costs 1.2 per cent when nights vary by a factor of one and a quarter, 2.3 at two and 2.0 at three; it beats the typical share from a factor of about 1.33 rather than 1.56, and loses to tonight’s estimate alone only at the widest spread, by a tenth of a point.

The best mixture moves with the spread of the nights, as it should. When nights barely differ the typical share is nearly right and deserves most of the weight: an eighth on tonight’s estimate at a factor of one and a quarter, a quarter at one and a half. When nights differ a great deal the typical share is nearly useless and tonight’s estimate deserves most: five eighths at a factor of two and three quarters at three. That is the structure of a shrinkage estimator, the same arithmetic by which a batting average early in a season is pulled towards the league’s, and it is the practical form of the warning in the weights are a guess the solve believes: a weight is only as good as what it was estimated from, and a blend is a way of saying how much that was. The spread of the nights plays the part of the spread of players, and the week’s pooling is what estimates it.

What a weather predictor would have to do #

The other route to tonight’s air is to measure it. The anomalous part of refraction near the horizon is driven by the temperature gradient in the lowest few metres of air, and the difference between the air and sea-water temperatures, which a ship can read, is the usual proxy for that gradient. Nothing here models that physics. What can be measured is how good a predictor would have to be: a share predicted from the weather, unbiased in its logarithm and explaining a stated fraction of the night-to-night variation.

With no skill it is the typical share and costs 10.3 per cent at a spread of two. Explaining half the variation it costs 3.7, still worse than tonight’s residuals at 3.2. It beats the residuals alone from about 54 per cent of the variation explained, and it beats the best blend of residuals and typical share — which is what a navigator should compare it with — from about 64 per cent. Below that, the sights themselves know more about tonight’s air than the thermometers do.

A predictor and the residuals are not rivals, and the natural next step is the obvious one. A weather predictor is a better prior than the typical share, and a blend that shrinks tonight’s residuals towards the weather’s prediction rather than towards the week’s average would beat both whenever the predictor has any skill at all. That is a combination the figure does not draw, because its value depends on a skill nobody here has measured.

How the estimate was checked #

Noise-free, the fit must return the share it was given. With every squared residual set equal to its expectation, the one-unknown regression returns shares of 0.05, 0.1 and 0.2 exactly, to $10^{-9}$.

The simulated fix error must be the fix error. For weights fixed in advance, forty thousand simulated fixes must reproduce the covariance formula’s mean-square error to within three per cent; they do. That is what licenses reading the data-dependent weights by simulation.

With no night-to-night change the typical share must win, because an estimate can only add noise to a weight that is already right. At a spread of one it costs nothing and tonight’s estimate 3.8 per cent.

With a large change tonight’s estimate must win, or there is nothing to find. At a factor of three the typical share costs 25 per cent on six thousand twilights and tonight’s estimate 2.2.

Every figure draws the same twilights. The predictor’s own random draws come from a separate stream, so asking for a predictor changes no twilight and no sight error, and the numbers in each caption are the same experiment seen from different sides.

Where the twilights stop #

The sextant’s error is taken as exactly known. In practice it would come from a week’s pooling and carry the uncertainty the earlier essay measured. A slightly wrong sextant variance moves tonight’s estimate by a fixed offset in $k^2$, which matters most on nights of small refraction error — the nights, as it happens, where a wrong estimate costs least.

The nightly factor is lognormal and independent from night to night. Real weather persists: a spell of settled conditions gives several similar nights in a row, and a share learned over the last three nights would then be a better prior than a week’s average. Persistence would favour pooling a short window and shrinking tonight towards it, and how much it helps depends on a correlation time this essay does not state.

The share is one number per night for every low body. A body in the direction of a warm current and one over cold water may see different refraction on the same evening, since anomalous refraction follows the air near the surface along each line of sight, and a share estimated from three low bodies on three bearings would then be an average of three different things.

The predictor is described by its skill alone. Whether any real measurement of the air–sea temperature difference explains 60 per cent of the night-to-night variation in the refraction error at 5° altitude is a question about marine meteorology that no simulation of sights can settle.

Still open: whether the refraction differs by bearing #

The last limitation is the one that could change the practical answer. Everything here assumed a single share for the evening, the same for every low body. If the refraction error near the horizon depends on the bearing — on whether the line of sight runs over warm or cold water, towards a front or away from it — then three low bodies on three bearings are three samples of three different shares, and pooling them into one estimate mixes the evening’s air with its geography.

A twilight has too few residuals to estimate a share per bearing; that is four numbers for three unknowns plus the fix’s own two. But a week has many, and a model in which the share varies smoothly with azimuth, with its amplitude estimated across the week and its value tonight shrunk towards that model, has the same structure as the blend measured here, one level deeper. Whether a navigator’s residuals over a week can see a bearing dependence of the size the air over a sea-surface temperature front would produce, and whether weighting by it is worth the extra parameter, are questions a single share for the whole sky cannot ask.