The same data on two grids

The cell ladder has spent five rungs inside one scheme. An address is an area; hexagons will not tile a sphere; a scheme trades area against shape; a query is a disc and a disc is not a cell; an address is a curve through the sphere. Every one of them takes the scheme as given.

Nobody has moved a number from one scheme to another, which is the operation that happens every time two datasets are combined — and which is not the same as reprojecting a raster, for a reason worth measuring rather than asserting.

Two schemes, and the overlap computed rather than sampled #

The two schemes here are the two anybody actually bins into.

Equal angle — equal steps in longitude and latitude — is what a plate carrée raster is and what almost every gridded dataset arrives as. Its cells collapse towards the poles, which is what an address is an area measures: a factor of 163 in area between the equatorial and polar cells at level 8. The address is a curve through the sphere then shows what that costs a database; this rung shows what it costs the numbers.

Equal area — equal steps in longitude and in the sine of latitude — is what anybody counting things per unit of ground uses, and is the cylindrical equal-area projection’s own grid.

Both are rectangles in longitude, so the overlap between a cell of one and a cell of the other is exactly computable: an interval intersection in longitude, an interval intersection in sin latitude, and the product times R². No sampling anywhere, which is what makes everything below a statement about the schemes rather than about a Monte Carlo. A rebinned value is the area-weighted mean of the source values over the target cell, which is the only definition that conserves anything.

The offset in longitude is not a complication invented for this essay. Two datasets built by different agencies will agree about the equator and about the prime meridian and about very little else: one bins from −180°, another from 0°, a third from the edge of its own coverage. Six degrees is a small disagreement and it is the one doing most of the damage below, because a cell boundary that falls in the middle of another cell is exactly the case averaging cannot survive.

What is conserved #

The total — the sum of value times area — survives exactly. Not approximately: to 2 × 10⁻¹⁶, which is the arithmetic’s floor, in both directions and after any number of round trips.

That is not an accident of these two schemes. Area-weighted rebinning is conservative by construction: every unit of area contributes its value to exactly one target cell with exactly its own weight, so the integral of the field over the sphere is unchanged by definition. Any implementation that fails this test has an overlap computation that does not partition the sphere.

And it is the only thing that survives.

What is lost #

The field’s own standard deviation is 0.370, so a single round trip introduces a per-cell error of 39 per cent of it, and moves one cell by more than one and a half standard deviations.

Nothing has gone wrong. Both rebinnings are exact and conservative; the loss is what averaging does. A target cell that straddles two source cells gets their mean, and the information distinguishing them is gone — permanently, because rebinning back cannot recover what was averaged away. The same irreversibility reprojecting a raster has, without any interpolation in it at all.

The decay is not exponential with a single rate, which a diffusion on a fixed grid would be. It is faster at first — 39 per cent of the variance in the first trip and about 15 per cent per trip thereafter — because the first pass removes everything at the scale of the cell mismatch and later passes work on what is left, which is smoother and harder to smooth further. The indicatrix is a limit has the same shape of argument: an operator’s effect is set by what it is given, and the first application is not like the tenth.

Rebinning is an averaging operator, and repeated averaging is a diffusion. A dataset that has passed through six pairs of schemes — which is an ordinary number for anything assembled from several sources and redistributed — has lost three quarters of its variance and has a completely intact total.

That is the shape of the failure and the reason it survives. Every conservation check passes. Every sum is right. Every regional total is right. The field is smoother than it was and nothing in the pipeline says so.

The number that would have caught it #

A validation check that compares totals cannot see any of this. One that compares variances can, and the ratio is a single number per rebinning: 0.61 for one round trip between these two schemes, and it is computable in advance from the schemes alone, without the data.

That is the practical output of this rung. Before moving a dataset between two schemes, rebin a stated test field through them and report the fraction of its variance that survives. It costs one pass over the overlap matrix, it is a property of the pair of schemes rather than of the data, and it turns a silent smoothing into a number in a log.

The equivalent for the resampling ladder is the convergence order, and the equivalent here is not an order at all — it is a ratio, because the operation has no limit to converge to unless both schemes refine together.

Where the damage falls #

The two schemes agree best where their cells are most alike, and an equal-angle cell and an equal-area cell are least alike where the equal-angle one has collapsed. At 83° the equal-angle cell is a sliver and the equal-area cell spans several rows of it, so the averaging is over a large and awkwardly shaped patch.

That is exactly the wrong place for the error to be. Polar data is scarce, expensive and the subject of the questions gridded data is most often asked — and it is where the operation nobody records is at its worst.

The asymmetry is worth naming. Going from the coarser scheme to the finer one loses almost nothing — a finer cell can be filled from a coarser one without averaging anything — and the loss is entirely in the other direction, where several source cells are combined. A round trip therefore costs exactly one coarsening, whichever order it is done in, and the scheme that is coarser in the region concerned is the one that sets the price. On these two schemes that is the equal-angle one near the equator and the equal-area one near the poles, which is why the error curve is not symmetric about the equator’s own value.

What this is, and is not #

It is worth separating this from the resampling the dataset ladder measures, because they look alike and are not the same operation.

Reprojecting a raster invents values interpolates: it asks what the field’s value is at a point, and answers with a weighted combination of nearby samples. It does not conserve anything, its error has a convergence order, and its kernels have names.

Rebinning integrates: it asks what the field’s mean is over a region, and answers with an exact area-weighted average. It conserves the total exactly, its error has no convergence order in the same sense — refining both schemes together reduces it, refining one alone does not — and there is only one right way to do it.

The two are confused constantly, because both are called resampling and both take a grid to a grid. The test that separates them is the total: an interpolating resample does not preserve it and a rebinning does. A pipeline that preserves the total is not thereby preserving the data, and a pipeline that does not preserve the total is not necessarily wrong — it may simply be answering the other question.

This is the same distinction computing an area needs a surface makes about what a number is a number of, arriving as an operation rather than as a quantity.

What was computed, and how #

The field is stated rather than sampled from anything: a sum of three sinusoids in longitude and latitude with structure at wavelengths of 120°, 51° and 33°, chosen so that a round trip has something to lose at more than one scale.

Each scheme’s cells are longitude intervals crossed with sin-latitude intervals, so their areas are R² Δλ Δ(sin φ) exactly. The overlap of two cells is the product of the two interval intersections, with the longitude intersection computed at three shifts of 2π so that a cell straddling the antimeridian is handled without a special case — which is the antimeridian being a cut in the numbers met in the one place it can be dealt with exactly.

The assertions have four parts. The total must survive to 10⁻⁹ — it survives to 10⁻¹⁶ — and the per-cell RMS must exceed 10⁻³, which is the refusal: an implementation that returned its input would pass the first and fail the second. Then the variance must fall over six trips, and the total must still not move after them.

The check that costs nothing #

The instruction is one line and it is the useful output of this rung.

Before moving a dataset between two cell schemes, rebin a stated test field through them both ways and record the fraction of its variance that survives. It is a property of the pair of schemes rather than of the data, so it is computed once and reused; it costs one pass over the overlap matrix; and it turns a silent smoothing into a number that can go in a log beside the file.

For the two schemes here that number is 0.61. Six of those in a row leaves 0.23. Nothing in a conservation check, a checksum or a total would show either, and the field would arrive looking exactly like a field.

Where the model stops #

Both schemes here are longitude-rectangular, which is what makes the overlaps exact. The interesting modern schemes are not: an HEALPix, an H3 or an equal-area cube has cells that are not rectangles in any coordinate, and their overlaps have to be computed by spherical polygon clipping or estimated by sampling. The conservation argument is unaffected — any exact overlap gives an exact conservation — but the numbers here are for the rectangular case and would be worse for schemes whose cells are less alike.

The field is smooth. A field with genuine discontinuities in it — a land mask, a category — is not merely damaged by rebinning but made meaningless by it, since the area-weighted mean of two categories is not a category. That case needs a majority rule rather than an average, and a majority rule conserves nothing at all. That is the categorical case a real edge has a width separates out for resampling, arriving here as a conservation failure rather than as a convergence one.

And nothing here is about time. Datasets are rebinned repeatedly across years as schemes change, and each generation inherits the previous one’s smoothing. The six round trips measured here are a model of that, and a poor one: real re-griddings alternate with new observations, which put variance back.

The generalisation #

Conservation and fidelity are different properties, and the first is much cheaper than the second. Any operation that partitions a domain and averages within the parts conserves the integral exactly and destroys structure below the part size. The integral is the quantity everybody checks, because it is the one with an obvious right answer.

This is the modifiable areal unit problem, which geography named in the 1930s and which is normally stated as the statistics depend on the zones. The form it takes here is sharper and more actionable: the statistic that is invariant to the zones is precisely the one that cannot detect the zones changing.

Who found it, and when #

The modifiable areal unit problem was named by Openshaw and Taylor in 1979 and identified by Gehlke and Biehl in 1934, who noticed that correlations between census variables rose as tracts were amalgamated. Conservative remapping in the exact sense used here comes from atmospheric modelling — the coupling of an ocean grid to an atmosphere grid, where a non-conservative flux is a physically wrong answer rather than an inaccurate one — and the standard algorithms are from the 1990s.

The two literatures do not cite each other. One is about statistics on administrative zones and the other is about mass fluxes between model grids, and they are the same operation with the same conservation property and the same information loss.

Two fields, one operation, no shared vocabulary #

That the statistics literature and the atmospheric modelling literature perform the same operation without citing each other is worth more than a footnote, because each has something the other lacks.

The modelling side has the algorithms and the conservation discipline. A coupler must conserve mass or the model is wrong in a way that accumulates, so the operation is implemented exactly — clipping cells on the sphere, weighting by true overlap area — and its conservation property is treated as a requirement to be tested rather than a hope.

The statistics side has the interpretation. Openshaw’s tradition knows what the information loss means for a conclusion drawn from the rebinned field: which statistics move, how far, and why a correlation computed on regrouped zones is not a correlation about the world.

Each field’s blind spot is the other’s subject. A modeller conserving mass perfectly has no vocabulary for the fact that the variance of the field has been reduced and that anything computed from it inherits that. A geographer worrying about the areal unit problem is often working with rebinning code that samples rather than clips, and so is carrying an error that the modelling side would call a bug on top of the effect they are studying.

The join is the sentence this rung ends on, and it belongs to neither literature: the invariant quantity is exactly the one that cannot detect the change. Conservation is what makes the operation defensible and is also what makes it undiagnosable from the conserved quantity — so a check on the total is guaranteed to pass and guaranteed to say nothing.

The absence of citation between the two is also self-perpetuating. Neither field’s search terms reach the other — conservative remapping and the modifiable areal unit problem share not one word — so a practitioner in either who went looking for prior work on their exact question would find their own literature and stop.

Which is why the check that costs nothing is a check on something else. Any second moment — a variance, a range, a count above a threshold — moves when the field is rebinned, and its movement is the size of the loss. That is the number the modelling side could compute and does not, because conservation is the property it was asked for.

It is also the reason this rung exists in a collection about maps rather than in either of them.

Where the ladder goes next #

Six rungs have addressed a cell, tiled a sphere with them, traded their area against their shape, queried them with a disc, ordered them along a curve and now moved data between two schemes. Every one has treated a cell as a fixed patch of ground. A cell system that is refined where the data is dense — which is what every operational scheme actually does — has cells of several sizes at once, and none of the measurements here survives that unchanged.