The class breaks were computed on the page

Three rungs of this ladder price what a reader does with a finished map — averages a colour, compares a symbol against a country, judges how thick a scatter of dots looks. All three failures live in the eye, and all three are therefore arguable at the edges: a reader is not an integrator, and the perception literature has an exponent for it.

This one does not live in the eye at all. A classifier is software. It takes a list of values, a list of weights, and a number of classes, and it returns boundaries. The weights are areas, the areas come from the geometry the machine is holding, and the geometry the machine is holding has been through a projection.

What a classifier is weighted by #

Every classification scheme in every mapping package offers a choice of rule — equal interval, quantile, natural breaks, standard deviation — and a choice of how to weight the observations. The default is by count: each region is one observation regardless of size. That default has a known problem, which is that it lets a thousand small units outvote three large ones, and the standard fix is to weight by area.

Weighting by area is right. The question this essay asks is which area, and the answer in practice is: whichever one is in the file.

A shapefile in a projected coordinate reference system carries projected geometry, and the area a geometry engine computes from it is a page area. The correction — carry the polygon back to the ellipsoid and integrate there, which is what computing an area needs a surface is about — exists in every serious library and is not the default in any of them, for the same reason a centroid belongs to the plane it was computed in: the planar formula is a shoelace and the spherical one is not.

The two classifiers #

Two rules are measured here, and they turn out to behave very differently, which is the finding.

Area-weighted quantiles. The boundaries are placed so that each class carries an equal share of the total weight. The weight is the definition, so a change of weighting moves every boundary by construction.

Natural breaks. The boundaries minimise the total weighted sum of squares about the class means. Computed here by the exact dynamic program over every contiguous partition rather than by Jenks’s iteration, so the answer is a global optimum and cannot come back different on a different day. The weight enters, but the boundaries are pinned to where the values have gaps, and a gap is a property of the data.

What was computed, and how #

The regions are the same 24-by-16 covering of latitude–longitude cells the whole anchor uses, with exact ground areas and shoelace page areas densified to forty-eight points a side. The fields are the stated closed forms from the first rung: a quantity rising steadily with latitude, a mid-latitude band, and a single northern concentration.

For each projection, each rule and each class count, the machinery computes both sets of boundaries and counts the regions whose class differs. The count is the measurement, because the class is the colour and a region on the wrong side of a boundary is drawn as belonging to a different group.

The boundaries for the latitude-varying field, on Mercator:

The page weighting has pushed the outer boundaries outwards by thirteen units on a field that runs from ten to ninety. The middle classes have swallowed the map. This is the arithmetic of giving the polar cells — 1.8 per cent of the Earth and 12.3 per cent of the sheet — nearly seven times their share of the vote in deciding where the quantiles fall.

How many regions change colour #

Half. A hundred and ninety-two regions of three hundred and eighty-four, from one decision in a geometry library that nobody made deliberately.

The equal-area rows are the control and they are stronger than the control the first rung had. There the equal-area projections returned a mean that came back right, which is one number. Here every individual region lands in the class it belongs in, on every equal-area projection in the library, under both rules and at every class count tried — because the weights are all multiplied by the same constant and a quantile of a distribution is unchanged by scaling every weight.

The classifier decides how much leaks in #

Quantiles are fragile and natural breaks are robust, and the difference is a factor of nine on the same map:

The mechanism is in the two definitions. A quantile boundary is defined by the weights: move a unit of weight and the boundary moves to keep the shares equal. A natural break is defined by where the values separate, and the weights only decide which of several candidate gaps is worth spending a boundary on. On a field with clear structure the same gaps win under either weighting, and the classification survives.

That is an uncomfortable ordering. Quantiles are the default in most software and the recommendation in most guides, for a good reason: they guarantee that every class gets used, which equal-interval schemes on skewed data do not. The property that makes them well-behaved with respect to the data is exactly the property that makes them badly behaved with respect to the weights.

What a producer can do, in order of cost #

The repairs are not equally expensive and they are not equally effective, which is worth laying out because the whole ladder ends in a recommendation.

Compute the areas on the ellipsoid. One function call in every serious geometry library, and it removes this failure entirely on any projection whatsoever. It is the only repair that is exact, it costs nothing at drawing time, and it leaves the map looking however the designer wanted it to look. Nothing in this essay argues for changing a projection; it argues for not taking a projected area as a ground area, which is computing an area needs a surface applied to a weight rather than to a measurement.

Use an equal-area projection. Also exact, and it repairs the three rungs below this one at the same time. But it constrains the picture, and a producer working to a house style or a fixed web basemap does not have the option.

Use natural breaks rather than quantiles. Cheap, partial, and unreliable — a factor of nine on the field measured here, and no guarantee at all on a field with no gaps in it. Worth knowing about as a mitigation and not as a fix.

Say which. A caption stating the projection and the weighting is one line, and it turns an invisible failure into a stated method. It is the same repair the ladder has now asked for three times, and it is the only one available to a producer who cannot change the pipeline.

The ordering is unusual for this collection in that the cheapest repair is also the best one. That is worth saying, because the reason the failure exists is not that it is hard to fix.

Why this is not the modifiable areal unit problem #

The obvious objection is that this is a known effect under a different name. It is not, and the separation is clean enough to state as a test.

The answer depends on the cells it was counted in prices what happens when the units change — slide a grid without changing its resolution and the largest reported value moves by 12.6 per cent. That effect is about which ground goes into which bucket. It would happen on a flat Earth with no projection at all, and it does not go away on an equal-area map.

This effect changes no unit and moves no boundary on the ground. Every region contains exactly the ground it contained. What changes is the weight each region carries into a statistic, and it vanishes identically on any equal-area projection while leaving the units alone.

The two can and do occur together, and a published choropleth generally has both. Telling them apart requires only asking whether the projection is equal-area: if it is, everything remaining is the areal-unit problem.

Where the model stops #

Equal interval and standard deviation are not measured here. Equal-interval breaks depend on the range alone and so are entirely immune — the weights never enter. Standard-deviation breaks depend on a weighted mean and a weighted variance and so should sit between the two rules measured, closer to the quantile end — and a weighted mean is exactly the statistic a choropleth is read by area prices, so its bias is already known. Neither is drawn.

The classifier is only half of what a package does. The colour scheme, the number of classes and the handling of outliers are all decisions with their own literature, and this rung holds all of them fixed to isolate one.

Real regions are not cells. Administrative units vary in size by orders of magnitude — which is the population a thousand features are wrong in the same direction is about, so an area weighting matters far more for them than for a regular covering, and both this effect and the areal-unit problem are correspondingly larger. The direction does not change: the projection’s contribution still vanishes on an equal-area map.

And the exact program is not what most software runs. Jenks’s iterative algorithm is a local optimiser that can land on a different partition depending on its initialisation, so a real pipeline has a third source of movement that has nothing to do with either weighting. Using the exact dynamic program here removes that source deliberately, so that everything measured is the weighting.

The generalisation #

The site’s standing rule is that an operation performed after the map is an operation performed on the map — a centroid, a buffer, a simplification, a strain rate.

Every one of those is a geometric operation, and that is what has made the rule feel like a rule about geometry: a centroid is a place, a buffer is a shape, a simplified line is a line. The general statement is broader and this rung is the case that shows it. A classification is not geometry at all. It is a partition of a list of numbers, its output is four scalars, and there is not a coordinate anywhere in the answer. It inherits the projection through a weight.

So the rule is about any quantity computed from a projected representation, and the ones with no geometry in the output are the dangerous members of the class — because there is nothing in a list of four boundaries to suggest that a coordinate system was involved in producing it.

One practical note about which weighting a package actually uses, since the rung turns on it. Most classification routines take a plain list of values with no weights at all, which is the count weighting and has its own well-known problem; area weighting has to be asked for, and asking for it is what brings the projected geometry in. So the failure measured here belongs to the more careful of the two available choices, which is the usual shape of a trap.

Who found it, and when #

The advice to use equal-area projections for choropleths is old and universal, and the areal-unit problem has been named and studied since Openshaw and Taylor’s work in the 1970s. Area-weighted classification is standard practice and is recommended precisely to stop small units dominating.

What appears to be missing is the join: the recommendation to weight by area, and the recommendation to use an equal-area projection, are made in different chapters, and the first quietly depends on the second. A pipeline that follows the area-weighting advice on a Mercator sheet has followed one recommendation into the arms of the failure the other exists to prevent, and every number it produces is plausible.

The dating of the classifiers themselves is not in doubt. Jenks published the natural-breaks method in 1967 and the exact dynamic program it approximates is Fisher’s, from 1958 — eleven years earlier, in a statistics journal, for a problem with no maps in it. Quantile classification is older than either and has no author worth naming. What all three share is that they were designed as partitions of a list of numbers, and a list of numbers has no coordinate system, so nothing in their statement warns that one has been supplied through the back door.

Which classifiers are exposed, and how much #

The two classifiers measured here are not equally at risk, and the ranking follows from what each one reads. It is worth setting out, because it tells a producer whose projection is fixed which choice is still theirs.

A quantile classifier by region count is not exposed at all. It sorts the regions by value and cuts the list into equal counts, and the areas never enter. Whatever the sheet does to the shapes, the same regions land in the same classes. The price is the one quantile classification always pays — a class may hold one enormous region and forty small ones — and that price is now the reason it is safe.

An equal-interval classifier is exposed only through its extremes. The breaks depend on the minimum and the maximum of the values and on nothing else, so the areal factor reaches them only if it changes which region is extreme, which it cannot: the values are attributes and the classifier is not weighting them. Equal-interval breaks are therefore stable under the projection too.

An area-weighted classifier is fully exposed, and that includes every method recommended for the reason area weighting is recommended: it reads the areas directly, and on a non-equal-area sheet those areas are the page’s rather than the ground’s.

So the exposure tracks the recommendation exactly backwards. The classifiers a textbook warns against — count quantiles, equal intervals — are the ones the projection cannot corrupt, and the one it recommends is the one it corrupts completely. That is not a coincidence: reading the areas is what makes a classifier good, and it is what makes it depend on which surface the areas came from.

The ordering also gives a producer a way to bound their exposure without changing anything: run the same data through a count-quantile classification and compare the assignments. Where the two agree, the weighting made no difference and the projection cannot have mattered; where they disagree, the weighting is doing work and the work is being done with the page’s areas.

Which gives the producer with a fixed projection a real choice rather than a counsel of despair. Either supply the ground areas to the weighted classifier explicitly — they are computable from the geometry on the ellipsoid, and the classifier does not care where its weights came from — or use a classifier that does not weight. The failure only happens when the areas are taken from whatever plane the geometry is currently sitting in, which is the default in every library and is nowhere written down as a decision.

Where the ladder goes next #

Four rungs have priced one thing four ways: a mean, a ratio, a density and a partition, each inheriting the areal factor by a different route, and each with an exactly zero on an equal-area map. What is missing is the case where the projection is not free to be equal-area — a national grid, a web map, an atlas sheet whose projection is fixed by somebody else — and the question there is not which projection to choose but how large a correction the fixed one requires and whether anybody applies it. That, and the reading a legend could carry to make the whole ladder unnecessary, are where this anchor goes.