Measuring distortion

The indicatrix is a limit

Tissot's ellipse describes an infinitesimal circle, and every published one is drawn finite. The error of the description falls as the radius rather than as its square — halving the circle halves the lie — and on the Robinson projection at a table entry it does not fall at all: sixty metres and six kilometres are both 1.17 per cent wrong.

Every serious treatment of map distortion says the same sentence: Tissot’s indicatrix is the image of an infinitesimal circle. Then it draws one about a centimetre across.

The word carries the whole of the construction. The indicatrix is the projection’s derivative — the linear map that takes a metre of ground into a displacement on the page — and a derivative describes a function exactly at one point and approximately near it. How near is a measurement nobody appears to make.

The image of a circle on Mercator, against its own indicatrix. A circle of three radii on the ground at 30°, 40°, projected exactly — the solid curve — against the ellipse the indicatrix predicts for it, dashed. The filled dot is the image of the circle's centre and the hollow one is the centre of area of what was actually drawn, which is not the same point. The departure runs from 3.14 per cent at 4° to 15.80 per cent at 16°, so it grows in proportion to the radius rather than to its square: halving the circle halves the relative error and does not quarter it.
Fig. 1 Three circles on the ground at 30° east, 40° north, projected exactly, against the ellipse the indicatrix predicts for each. The solid curve is where Mercator actually puts the ring of points; the dashed one is where the derivative says they go. At 4° of radius the two are within a fifth of a per cent; at 16° the gap is visible without measuring. The hollow dot is the centre of area of what was drawn and the filled dot the image of the circle’s own centre, and they are not the same point.

The quantity, and why it is a ratio

The obvious measurement is the distance between the true image of a point and where the linear map put it. That number is in page units, which are arbitrary — a projection returns radians, a plotting library multiplies by whatever fits the column — so it says nothing on its own.

What means something is the departure relative to the size of the ellipse being drawn. The indicatrix at a point has semi-axes aa and bb; a circle of geodesic radius ρ\rho is drawn as an ellipse of mean semi-axis 12(a+b)ρ\tfrac{1}{2}(a+b)\rho; and dividing the worst departure by that gives a pure number, which is the fraction of the picture the ellipse gets wrong.

Nothing is fitted anywhere in this. The predicted ellipse is not an ellipse chosen to match the image — it is the Jacobian the site differentiates for every other quantity it computes, applied to a ground displacement of ρ\rho in each direction in turn.

The law, and it is not the one that was expected

The absolute departure is second order in the radius. That is what “the indicatrix is the derivative” means and it is what any account of the construction implies.

The relative departure is first order, because a quadratic divided by the ellipse’s own linear size leaves something linear. Both exponents are fitted here rather than asserted, and each is required to reject the other’s law:

radius departure as a fraction of the ellipse
13 km 2.2 × 10⁻⁶ 0.084 %
32 km 8.8 × 10⁻⁶ 0.211 %
64 km 3.5 × 10⁻⁵ 0.424 %
127 km 1.4 × 10⁻⁴ 0.855 %
319 km 5.8 × 10⁻⁴ 2.202 %
637 km 2.4 × 10⁻³ 4.632 %

Four times the radius is sixteen times the absolute departure and four times the relative one. The fitted exponents are 2.016 and 1.016 across five projections, and a linear law fits the first no better than a quadratic fits the second.

The practical consequence is worth stating plainly, because the intuition runs the other way. Halving the region halves the error the indicatrix makes about it. A second-order effect would be quartered, and a reader who expects second order will believe an indicatrix twice as far as it deserves.

How the departure shrinks with the circle, at 30°, 40°. The departure of a circle's image from the ellipse the indicatrix predicts, against the circle's radius, on logarithmic axes. A slope of one is the first-order law: relative to the size of the ellipse being drawn, halving the circle halves the error rather than quartering it. Mercator has slope 1.01; Albers equal-area conic has slope 1.00; Robinson has slope 0.11. A flat line is a projection whose derivative jumps at this latitude, so the departure does not shrink at all and the limit Tissot's construction describes does not exist there.
Fig. 2 The departure against the radius on logarithmic axes, at the same point, for three projections. Mercator and the Albers conic lie on lines of slope one — the first-order law — separated by the rate each one’s scale factor changes at. The Robinson projection’s line is flat, and that is not a small slope; it is the absence of a limit, and the next section is about it.

What the ellipse is, before it is asked how far it reaches

The two numbers the indicatrix carries are the semi-axes, and they are computed at a point by a decomposition with no circle in it at all: build the matrix that takes a metre east and a metre north into a page displacement, and its singular values are aa and bb. That is the object Tissot’s indicatrix draws and what survives a change of coordinates shows is a property of the map rather than of the graticule.

The indicatrix at 30°, 40° on Mercator. A circle on the sphere becomes this ellipse on the map. Its semi-axes are the two principal scale factors, a = 1.305 and b = 1.305; their product is the areal factor 1.704; and the maximum angular deformation is 0.00°. h and k are shown too, and depend on the coordinates rather than on the map.
Fig. 3 The ellipse at the same point, with its axes and its numbers. On a conformal projection the two semi-axes are equal to the noise floor of the differentiation, so the ellipse is a circle and the whole of the distortion is in its size. Nothing in this construction refers to a circle of any particular radius; that is what makes the question of how large a circle it describes a separate question with its own answer.

The distinction matters because the two quantities behave differently across a map. The semi-axes at 40° on Mercator are both 1.305, and they would be the same numbers whatever radius the drawing used. The honesty of the ellipse at that point is a third number, and it is not derivable from the first two.

The rate has a closed form, on the projection where the algebra is short

The coefficient of that first-order law is not a free constant. Expanding Mercator’s northing about a point gives a second derivative of secφtanφ\sec\varphi\tan\varphi against a first derivative of secφ\sec\varphi, and the ratio of the two is the whole of it: the relative departure of a circle of radius ρ\rho is

12ρtanφ\frac{1}{2}\,\rho\,\tan\varphi

with nothing left over. Measured against that prediction at 15°, 30°, 45° and 60°, the worst discrepancy is 0.14 per cent, which is the residual third-order term.

That formula says what the coefficient is in general, even where the algebra is long: it is the logarithmic rate at which the projection’s own scale factor changes with position. A projection whose scale is nearly constant over a region has an indicatrix that stays honest across it; a projection whose scale is changing quickly has one that does not, regardless of how large the distortion itself is.

The two are independent, and separating them is the point of the measurement. Mercator at 40° has an areal factor of 1.70, which is enormous, and an indicatrix good to a per cent out to 148 kilometres. The gnomonic projection 20° from its centre has a smaller areal factor and an indicatrix good to 46.

How large a circle each indicatrix actually describes

Inverting the question gives the number a reader can use: at a stated tolerance, how big may the circle be? Bisection rather than the first-order law, because the law is what the other figure is testing.

How large a circle each indicatrix describes, at 30°, 40°. The radius at which the image of a circle departs from the ellipse Tissot's construction predicts by 1 per cent, measured by bisection at 30°, 40° on eight projections. Albers equal-area conic keeps its ellipse honest to 829 kilometres and Gnomonic to 46 — a factor of 17.8, decided by how fast each projection's own scale factor changes rather than by how large its distortion is.
Fig. 4 The radius at which each projection’s indicatrix is one per cent wrong, at 30° east, 40° north. The Albers conic keeps its ellipse honest eighteen times further than the gnomonic does, and the order has almost nothing to do with how distorted each projection is at that point — it is the order of how fast each one’s scale factor is changing there.

At one part in a thousand — a tolerance a survey might recognise — every one of these numbers divides by ten, because the law is first order. The gnomonic’s ellipse is then a fair description over five kilometres, and Albers’ over eighty-five.

The dependence on latitude follows the same closed form, and on a cylindrical projection it is severe: tanφ\tan\varphi is zero at the equator and unbounded at the pole.

Where the indicatrix stops describing the circle, on Mercator. The radius of the largest circle whose image is within 1 per cent of the ellipse Tissot's construction predicts, across latitude on Mercator. It runs from 1549 kilometres at 0° to 34 at 75°, because the rate the departure grows at is the rate the projection's own scale changes — and on a cylindrical projection that is ½ tan φ, which vanishes on the equator and runs away at the pole.
Fig. 5 The one-per-cent radius on Mercator against latitude. It is 1,549 kilometres at the equator, where the projection’s scale is momentarily stationary, 214 at 30°, and 34 at 75°. An indicatrix drawn at the same size at every latitude on a world map is a fair description of a very different amount of ground in each place.

The image of a circle is in the wrong place as well as the wrong shape

The departure measured above is of shape. There is a second measurement in the same picture and it is the one that connects this essay to the operations field.

The image of a circle is not centred on the image of the circle’s centre. The second-order term is not symmetric about the point, so the centre of area of the drawn ring sits away from the projected centre — by 0.084 per cent of the ellipse at 13 kilometres and 4.2 per cent at 637, which is the same first-order law with a slightly smaller coefficient.

That is exactly the defect a centroid belongs to a plane measures at country scale, arriving here at the scale of an indicatrix and from the opposite direction. A centroid computed in a projected plane is not the centroid of the ground it came from, and the reason is visible in a picture six pixels across.

Where the limit does not exist at all

The whole account above assumes the projection has a second derivative. One member of the site’s library does not.

The Robinson projection is defined by a table of numbers at five-degree intervals — it is not an approximation to a formula, because there is no formula — and interpolating it linearly makes the first derivative a step function. At a table entry the derivative jumps, and the second derivative is a spike.

The consequence is not that the departure is large. It is that the departure does not shrink:

radius at 40° N departure
60 m 1.174 %
640 m 1.174 %
6.4 km 1.175 %
64 km 1.294 %

A hundredfold reduction in the circle leaves the error where it was. Tissot’s construction describes the limit of a shrinking circle, and at a table entry on Robinson there is no limit to describe: the ratio the definition takes a limit of simply does not converge.

Two degrees away, between entries, the projection is a polynomial and behaves ordinarily — 0.005 per cent at 640 metres, 0.045 at 6.4 kilometres, 0.453 at 64. Perfectly first order. So this is a statement about the table and its interpolation, not about the shape Robinson drew.

The site’s own implementation comments that a cubic interpolation would be closer to Robinson’s intent and that the difference is “well below the width of a drawn line”. For the drawn map that is true. For this quantity it is not true at all, and the measurement is the only thing that would have said so.

What this does to a published figure

Almost every indicatrix in print is drawn at a size chosen so the ellipses are visible — a few degrees of arc, sometimes ten or fifteen, scattered across a world map at twenty-degree intervals.

At 4° of radius on Mercator at mid-latitude the ellipse is right to about a fifth of a per cent, which is far better than the drawing’s line width and entirely fine. At 16° it is wrong by around four per cent, which is a visible discrepancy in a figure whose whole content is a shape. And at the sizes some textbook figures use — circles large enough to touch each other — the ellipse is describing something the projection does not draw.

This site’s own indicatrices are drawn at a fixed size in page units rather than in ground units, which is a different decision and has to be said out loud: Tissot’s indicatrix draws each ellipse from the measured semi-axes at a point, at a size chosen for legibility, and does not claim that a circle of that ground radius maps to it. The numbers are the content and the ellipse is their picture.

The same failure, with a triangle instead of a circle

A circle is not the only finite object an infinitesimal statement gets applied to. The other one is a triangle, and it fails in a way that is easier to see and harder to dismiss.

A triangle of 25.0° excess, drawn on Mercator. The three geodesic sides as curves and the three straight sides a ruler draws, with each vertex labelled by how far the ruler's angle is from the true one. The straight-sided triangle's angles sum to exactly 180° because it lies in a plane, and the real one's sum to 205.02°, so the three errors have to account for the whole 25.02° of spherical excess between them — and they do so on this conformal projection exactly as they do on any other.
Fig. 6 A triangle drawn between three places on a conformal projection, with its three angles measured on the page and on the sphere. Conformality is a statement about the angle between two curves at a point; a triangle’s corner is at a point, but the sides that meet there are straight lines on the page and geodesics on the ground, and those are different curves. The angles disagree by degrees on a projection that preserves angles exactly.

That is conformal does not mean the angles are right and it is the same mistake as reading a finite circle off an indicatrix: an infinitesimal property, exercised at a size where the infinitesimal has stopped applying. The two essays measure the two shapes, and the pattern they share is worth naming — a pointwise property is a promise about a neighbourhood whose size is never quoted.

The relation to flatness, which is the same question asked once further out

How small is flat enough asks what error a plane makes about a patch of sphere, and finds it quadratic in the patch’s size. This asks what error the derivative of a projection makes about a patch, and finds it linear once the ellipse’s own size is divided out.

They are the same expansion truncated at different places. The flat-Earth error is the second-order term of the sphere, with no first-order term available to divide by; the indicatrix error is the second-order term of the projection, relative to the first-order term the indicatrix already carries. A quantity measured against a leading term that is itself proportional to the size loses one power.

The pair is worth holding together because the two numbers are so different in practice. A plane is good to one part in a million over ten kilometres; an indicatrix is good to one part in a thousand over fifteen. The projection’s own variation is a far cruder thing than the curvature of the Earth.

How large a patch can be treated as flat, at 10 parts per million. The smallest scale distortion any map of a circular patch can have, against the radius of the patch, on logarithmic axes. The line is straight with a slope of 2.00: the error grows as the SQUARE of the size, so a patch ten times wider is a hundred times worse. A tolerance of 10 ppm is reached at a radius of 40.3 km — 81 km across — and that is the number behind the boundary between plane surveying and geodesy.
Fig. 7 The other truncation, for comparison. Treating a patch of the Earth as flat costs an error that grows as the square of the patch, so a stated tolerance buys a radius that falls as its square root. The indicatrix’s error grows as the first power of the radius, so a stated tolerance buys a radius that falls in proportion — which is the harsher of the two laws and applies to the smaller distances.

Where the rate is largest, and why it is not where the distortion is

The rate is a derivative of the scale factor, so it is largest where the scale is changing fastest — which on a cylindrical projection means near the poles, and on an azimuthal one means at the rim.

How Mercator distorts, by latitude. Angular deformation and areal scale plotted against latitude along the meridian at 0°. On Mercator the angular deformation reaches 0.0° and the areal factor reaches 91.5.
Fig. 8 Mercator’s own scale factor against latitude. The indicatrix’s honesty is governed by the slope of this curve rather than by its height: at the equator the curve is flat and the ellipse is a fair description out to 1,549 kilometres, while at 75° the curve is climbing steeply and the same description survives 34.

A projection can therefore be very distorted and very well described, or nearly undistorted and badly described. The Albers conic at 40° is close to true scale and has the honest indicatrix of the eight tested; the gnomonic 20° from its centre is moderately distorted and has the worst. Nothing about the size of the distortion predicts either.

The measurement’s own controls

Three, and each catches a different way this could have been a fact about the machinery rather than about the projections.

The circles are geodesic circles on the sphere, generated from the destination formula at a fixed distance and a sweep of azimuths, not from a small square in longitude and latitude. A ring at constant Δφ\Delta\varphi and Δλ\Delta\lambda is not a circle on the ground at all, and using one would measure the graticule’s own anisotropy and report it as the indicatrix’s error.

The predicted ellipse is the site’s own Jacobian, Richardson-extrapolated from central differences, the same object what survives a change of coordinates shows the invariants are built from. If it were a separately fitted ellipse the measurement would be of the fit.

Both exponents are required, and each rejects the other. A check that only demanded the relative departure be small would pass on a projection that was linear everywhere; one that only demanded second-order absolute growth would pass while the useful statement — what happens to the drawn ellipse — went unmade.

What the varying radius asks of a sampling scheme

The radius over which an indicatrix honestly describes its neighbourhood is a function of position, and that turns the usual way a regional criterion is computed into something that needs justifying rather than assuming.

A regional score is a sum over a lattice. Sample the distortion at a few hundred or a few thousand points, weight them, and total. The lattice spacing is chosen for cost, and nothing in the criterion’s statement says what it should be.

The honesty radius says what it should be. Each sample describes a patch of the size over which its own linearisation holds, and the samples should tile the region rather than overlap or leave gaps — so the spacing wants to be comparable to the local radius. Where the radius is large the sampling can be coarse and the extra samples buy nothing; where it is small the sampling must be fine or the sum is of descriptions that do not reach each other.

And the radius varies by orders of magnitude across a world map, so a uniform lattice is necessarily wrong somewhere. It is wasteful in the well-behaved middle of a projection and insufficient near the places the projection is doing something — which are the places that decide the score.

The adaptive version is not difficult. The radius comes from the same second derivative the criterion’s own machinery computes; scaling the local spacing by it is one more pass, and the result is a sum whose terms describe disjoint patches of ground rather than an arbitrary set of points.

The check, for anybody unwilling to change their sampling, is the ordinary one: refine the lattice and see whether the regional score moves. A score that is stable under refinement is summing patches that already tile the region. A score that drifts is being computed at a spacing the projection does not permit, and the drift says in which direction.

It also gives a reader a way to judge a published regional comparison they cannot rerun: ask what the sampling was, and whether it was uniform. A uniform lattice on a world map is a declaration that the criterion was evaluated at a spacing chosen for convenience.

None of this is an objection to the criteria themselves. It is a statement about the resolution at which they are meaningful, and the point of computing the radius is that the resolution stops being a matter of taste.

Where the ladder goes next

The indicatrix has now been asked what it is (an ellipse with two axes), what it says (two independent failures), which parts of it survive a change of coordinates, where its worst point is, which way it points, and how large a circle it describes.

What remains on this ladder is the step from a point to a region, which distortion over a region begins and does not finish: integrating a pointwise quantity over a country requires a weighting, and the weighting is where measurement stops and judgement starts. This essay adds a constraint to that integration nobody states — the integrand is only a description of the ground over a radius that is itself a function of position, so a regional criterion sampled on a coarse grid is summing descriptions of overlapping patches of varying honesty.

That is not a defect in the criteria. It is a statement about the sampling density they need, and it now has a number attached to it.

Named alongside this one

Essays reaching for the same objects. Nobody chose these; they are what the concept index makes visible.

What links here

The 8 essays that link to this one and share the most of its objects, of 16 that link here.

The objects this essay names

Each one links to every other essay that touches it.

CentroidInfinitesimalJacobianLimitNumerical differentiationPrincipal scale factorsQuadratic lawRobinsonSeries truncationTissot's indicatrixToleranceVerification