Further north on the grid is not further north

A grid coordinate is a pair of numbers, and thirteen rungs of this ladder have asked what those numbers refer to: where the origin is, what the units are, what a reference names, how wide the zone should have been.

A pair of numbers is also an ordering. North of is decided by comparing two northings, and comparing two numbers is the cheapest operation in any database. That relation is not the relation on the ground, and nothing about a grid says so.

The mechanism, in one line #

A transverse Mercator’s northing is not a function of latitude alone. Away from the central meridian it picks up a term in the square of the longitude difference,

$$N \approx M(\varphi) + \tfrac{1}{2}\,\nu \sin\varphi\cos\varphi\,\Delta\lambda^2,$$

which is the same term that makes grid north differ from true north. There the consequence is an angle a surveyor has to correct a bearing by. Here it is a comparison a computer performs without any correction at all.

The same quantity, in the form the ladder already had it #

The convergence is not new to this collection, and it is worth putting the familiar picture beside the unfamiliar one.

The two shapes are different because the two quantities are different. The angle goes as sin φ and grows towards the pole. The latitude difference the grid reverses goes as sin φ cos φ, because the northing a parallel gains is an angle times a distance and the distance is a parallel’s own length, which carries the cos φ. So the angle is worst at the pole and the ordering is worst at forty-five degrees — and the share of pairs affected, below, goes back to following the angle.

Three quantities, one term, three different shapes in latitude. Reading any of them off the others is how a plausible mistake gets made.

How large it is #

Divide the northing a parallel gains across the zone by the northing a degree of latitude buys, and the answer is a latitude difference: the largest one the grid can reverse.

The peak is at forty-five degrees and it is not a coincidence of the ellipsoid: sin φ cos φ is half sin 2φ, which is largest where the sine and the cosine are equal. So the worst latitude for this failure is the one most of the world’s mapped land sits near, and the two ends of the axis — the equator and the pole — are the two places it does not happen at all.

Four and a half kilometres is not a subtlety. It is two towns, or the two ends of a city, and a list sorted by northing will put them the wrong way round.

How often it happens #

A largest case is not a rate. The rate needs a population, and the population has to be pairs rather than points, because an ordering is a relation: one place drawn anywhere in the zone, the other at a stated distance from it on a random bearing.

Two things in that figure are worth separating.

The share is about one in two hundred, which is small and is not negligible: a national dataset of a hundred thousand pairs of neighbouring features contains five hundred that a northing comparison orders wrongly.

And the share does not depend on the separation. That is the surprising half, and it is a shape no error budget written in metres would have predicted. Half a kilometre apart or a hundred, the answer is the same 0.54 per cent — which rules out the obvious mental model, in which the effect is a fixed few kilometres of error that matters for nearby places and washes out for distant ones. It does not wash out, and adding data at a coarser scale does not dilute it.

Why it is scale-free, and the prediction that follows #

The reason is that both quantities scale together. A pair at separation d on bearing α has a latitude difference proportional to d cos α and a convergence-induced northing difference proportional to d sin α, so the ratio that decides the flip has no d in it at all.

What is left is a statement about bearings. A line of constant northing runs at 90° + γ rather than at 90°, where γ is the meridian convergence, so the pairs the grid reverses are exactly those whose bearing falls between the two lines — a band of width |γ| on each side of the compass. Over uniformly drawn bearings the share is

$$\frac{|\gamma|}{180°}.$$

There is no fitted constant in that. The mean convergence over the zone is computed from the projection, divided by 180, and compared with a count — and the two agree to seven per cent, with the residual being the difference between a uniform distribution of longitudes and the one the population actually has.

The convergence is therefore two things at once. It is an angle a surveyor adds to a bearing, which is what every manual says it is, and it is a probability that a comparison of two coordinates comes out backwards. The second use has the same number in it and nobody writes it down.

The east ordering, which is worse #

The relation nobody suspects fails more often than the one this essay is named after.

Easting on a transverse Mercator is approximately ν cos φ Δλ, so it carries a factor of the cosine of the latitude. Two places at different latitudes have their longitude differences scaled differently before they are compared, and the comparison can come out backwards. Over the same pairs the east ordering is reversed 0.59 to 0.68 per cent of the time — consistently more often than the north one.

Over pairs drawn independently rather than at a fixed separation, where the latitudes can differ by tens of degrees, the east ordering fails on 4.3 per cent of pairs. That is the population a query over a whole country actually has.

Why the east ordering fails differently #

The east relation is reported as worse and the reason is not that the convergence is larger there — it is the same convergence. The east ordering has a second mechanism that the north ordering does not, and the two respond to different things.

The northing’s error is purely the convergence term: a line of constant northing runs at 90° + γ instead of 90°, and the share of pairs it reverses is the width of that band over a half-turn, γ̄/180°.

The easting carries an extra factor. It is approximately ν cos φ Δλ, so two places at different latitudes have their longitude offsets scaled by different cosines before being compared. Over a dataset spanning 30° to 60° north, cos φ runs from 0.866 to 0.500 — a factor of 1.73 — so a place further east in longitude can be drawn further west on the grid whenever its offset from the central meridian is less than 1.73 times the other’s and it sits at a higher latitude.

That mechanism has nothing to do with the zone’s width. It depends on the latitude span of the data, which is why the east ordering fails on 0.6 per cent of pairs at a fixed small separation — where the two latitudes are nearly equal and the cosines nearly cancel — and on 4.3 per cent of independently drawn pairs, where they do not.

Three consequences follow, and the third is the one a designer needs.

The two failures have different cures. Narrowing the zone reduces the convergence and therefore the north ordering’s share, in proportion. It does nothing at all to the cosine term, so a narrow national grid is safer for north of and exactly as unsafe for east of.

The east failure grows with the dataset rather than with the map. A query over one county has a latitude span of a fraction of a degree and a cosine ratio of 1.005; a query over a country spanning thirty degrees has 1.73. The same grid, the same code, and a share that grows by nearly an order of magnitude as the region of interest widens.

And it is the relation nobody suspects. North of at least sounds like it might involve a projection, because northing sounds like latitude. East of sounds like a comparison of longitudes, which is what a reader believes an easting is — and it is a comparison of longitudes each multiplied by a different number.

Where this bites, and it is not surveying #

A surveyor knows about convergence; it is on the margin of every sheet, and correcting a bearing for it is a step in every traverse. The places this reaches are the ones where nobody is thinking about a projection at all, because the operation does not look geometric.

A sort. Order by northing descending is the natural way to list features from north to south, and it is wrong on one pair in two hundred.

A bounding box. A query for everything north of a line is a comparison against a northing, and it returns a set whose boundary on the ground is not the line asked for but the line of constant northing — which climbs 4.4 kilometres of latitude across the zone. That is the same failure a query is a disc and a disc is not a cell prices for a different query shape.

And a join between two zones. Two datasets in adjacent UTM zones have their orderings taken about different central meridians, so a feature can be north of another in one zone’s numbers and south in the other’s — which is what happens where two zones meet, arriving in a relation rather than in a coordinate.

What a fix costs #

Three repairs, in increasing order of what they cost and decreasing order of how often they are done.

Compare latitudes. The ordering the reader means is the ordering of the geodetic latitudes, and the inverse projection is available. It costs one inverse per row, which on a modern machine is nothing and on a database index is everything — an index is built on the stored number, and the stored number is the northing.

Correct the northing. Subtracting ½ ν sin φ cos φ Δλ² from each northing before comparing recovers the meridian arc, which is monotone in latitude. That is a scalar correction per point, it can be stored alongside, and it is exactly the reduction a surveyor performs when going from the grid to the ellipsoid.

Or narrow the zone. The share falls with the mean convergence, so a national grid one degree wide reverses a tenth as many pairs as a six-degree one. That is a real argument for a narrow zone and it is not the argument the zone-width rung makes — that one is about scale error, which is a different function of the width, and the two would design different systems.

The repair nobody should take is a tolerance: refusing to order pairs whose northings differ by less than some threshold. The threshold that would work is 4.4 kilometres of latitude, which is larger than most of the pairs anybody wants ordered.

The relation a reader actually holds #

It is worth separating three things a reader might mean by north of, because the grid answers only one of them and the other two disagree with it in different ways.

Greater latitude is what a geographer means, and it is what the essay above compares the grid against. It is a statement about the ellipsoid.

Further along a meridian is what a traveller means, and it is the same relation: a meridian is a curve of constant longitude and increasing latitude, so travelling north increases latitude.

Higher up the page is what a reader of a printed sheet means, and it is the grid’s relation, because a sheet is laid out with northing up. That third one is the only one a database can compute without a projection library, and it is the only one that is not about the ground.

The three coincide exactly on the central meridian and nowhere else. What makes the failure survive is that the three names sound like synonyms, and on a map of a town — where the whole sheet is within a few kilometres of one meridian — they are.

The repair is not to fix the ordering, which is correct for what it computes, but to stop the phrase. A query that means greater latitude should say so and should be answered on the ellipsoid; a query that means higher on the sheet should say that instead, and is then a statement about a page rather than about the world.

Where the model stops #

The zone here is a plain six-degree transverse Mercator with UTM’s scale factor. A national grid with a different width and a different origin has a different convergence and the same law: the share is its own mean convergence over 180 degrees, and a narrow national zone is correspondingly safer.

The population is uniform in longitude across the zone and in latitude between 30° and 60°. A real dataset is neither, and the prediction is a mean over a distribution that a real one will not have — which is why the prediction is stated as γ̄/180 rather than as a number.

The measurement is of the map, not of the data. A coordinate also has a width, and two places whose northings differ by less than that width are not ordered by the grid in any meaningful sense at all. At a metre of precision that removes pairs closer than about ten metres in latitude and nothing else; at a kilometre of precision it would be most of the effect.

The generalisation #

A coordinate system supports more operations than the one it was designed for, and a projection is chosen against one of them.

A transverse Mercator is chosen because it is conformal and because its scale error is small over a narrow zone — which is what a zone is for. Nothing in that choice is about ordering, and ordering is what a database does with the coordinates all day.

The pattern recurs across this collection: a centroid belongs to a plane, an area on the grid is not an area on the ground, nearest is a question about the metric. Each is an operation performed on the coordinates rather than on the ground, and each is wrong by the amount the projection is not the identity. This one is the cheapest operation of them all, which is why it is the one nobody checks.

The asymmetry between the two relations also explains why the fix list ends where it does. Comparing latitudes repairs both; correcting the northing repairs only the north relation, because the cosine term is not in the northing at all; and narrowing the zone repairs the north relation in proportion and the east one not at all. Only the first of the three is a repair rather than a mitigation, and it is the one an index cannot use.

Who found it, and when #

Convergence has been on the margin of every gridded sheet since the national grids of the 1920s and 1930s, and the formula γ ≈ Δλ sin φ predates them by a century — it is the first term of the transverse Mercator series that Gauss and Krüger developed.

The consequence for ordering belongs to the database era and is, as far as this collection can find, unwritten. It is implicit in every warning that geographic queries should be done in geographic coordinates, and that warning is usually given about distance and area rather than about comparison, because distance and area are obviously geometric and greater than does not look geometric at all.

Where the ladder goes next #

Everything on this ladder holds the grid fixed and asks what it does to a coordinate. The grid is a set of marked points on ground that is moving, and the interval before a network breaks its own tolerance is the question this one has never asked.