Concept

Ellipsoid — where it appears

The surface of revolution a datum uses as its model of the Earth, defined by a semi-major axis and a flattening and by nothing else. Its flattening is not a free parameter — Clairaut's theorem ties it to the body's spin and its dynamical form factor, so it can be checked rather than assumed.

Named by 38 essays across 7 fields — each of them below, with the objects they name alongside it.

The same coordinate on four datums. One pair of numbers — 2.0° west, 54.5° north — read as a coordinate on OSGB36, ED50, NAD27 and on WGS84, and plotted where each reading puts the mark on the ground. The spread runs to 195 metres. The numbers are identical; only what they refer to differs.

What a coordinate refers to

A latitude and a longitude are not a place. They are a claim about a model, and the model has a shape, a position, a set of marked stones and a date — change any one of the four and the ground under the numbers moves by hundreds of metres.

datums · Datum
Every projection in the library, measured against both properties. Maximum angular deformation across the bottom, maximum areal error up the side, both on logarithmic scales and both measured over several hundred sample points rather than taken from the projection's description. Conformal projections lie on the left edge, equal-area ones along the bottom, and the corner where both vanish is empty because a projection there would be an isometry of the sphere onto the plane. All 26 are plotted; the four named here are Mercator, Mercator (ellipsoidal), Web Mercator, Gall–Peters.

Web Mercator is not conformal

It carries almost every map on the internet, it is named after the projection whose entire purpose is preserving angles, and it does not preserve angles. The machinery here found that without being told to look.

wrong · Audit
OSGB36 to WGS84, one parameter at a time. Each bar is how far the mark at 2.0° west, 54.5° north moves under one of the seven parameters with the other six set to zero. seven of the seven are not zero for OSGB36. The units hide the comparison: one arcsecond of rotation moves this point 30.8 metres and one part per million of scale moves it 6.36 metres, so OSGB36's scale term contributes 130 metres — more than one of its three translations.

The seven parameters, and what each one does

A datum transformation is published as three metres, three arcseconds and one part per million, in three different units — which hides the fact that all seven are the same size of thing. Converted to ground displacement, the scale term beats two of the translations.

datums · Datum
Fitting a sphere to 10° of latitude at 54.5°. The Earth's Gaussian radius of curvature across the band, with two candidate spheres drawn against it. The global mean radius is 6371.0 kilometres and sits 2203 parts per million away from the ground here — 2.2 metres in every kilometre measured. The best local radius is 6385.0 kilometres and has no bias at all by construction, leaving 325 parts per million of spread that no sphere can remove, because the curvature varies across the band and a sphere's does not. The gain is a factor of 6.9.

A datum is fitted to a region

Forty countries adopted forty ellipsoids, and the usual explanation is that measurement was poor. It was not — a figure fitted to one country beats the global one over that country by a factor of seven, and what it removes is a systematic bias of two metres in every kilometre.

datums · Datum
Why there are two latitudes, at 45°. A meridian section of an ellipsoid with the flattening exaggerated 12× — at the true value of 1/298 this outline would be indistinguishable from a circle. Geodetic latitude is the angle the surface normal makes with the equatorial plane; geocentric latitude is the angle the radius makes with it. The two lines meet the plane at different points and the angles differ by 11.55 arcminutes on the real WGS84 ellipsoid, which is about 21 km of ground distance.

The Earth is a sphere, and when it is not

Every essay before this one treated the Earth as a ball, and said so. The flattening is one part in three hundred, which is nothing for a distance, everything for a latitude, and exactly enough to make the most-used projection in the world fail the property in its own name.

wrong · Ellipsoid
Transverse Mercator. The graticule of the Transverse Mercator projection at 30° of longitude and 15° of latitude. the only projection on which a constant compass bearing is a straight line. It is conformal.

Transverse Mercator and the series that computes it

The projection most of the world's survey data lives in has no closed form. What every national grid actually computes is a truncated power series in the flattening, and how far it can be trusted is an engineering parameter rather than a property of the projection.

families · Ellipsoid
London to Tokyo on Orthographic. Two routes. The great circle is 9559 km and is the shortest path on the sphere. The rhumb line holds a single compass bearing the whole way and is 11296 km — 1737 km further, or 18.2 per cent. On Orthographic the rhumb line departs from straight by 2.9e-1 of its own length.

The great-circle vertex

One quantity is constant along a shortest path on a sphere, and it fixes the highest latitude that path will reach before the journey starts. That number is why polar routes exist, and it can be read off the departure bearing without tracing the route at all.

paths · Paths
The sixty zones, each six degrees wide. Every zone is a separate transverse Mercator projection about its own central meridian, so the world is covered by sixty maps rather than one. Zone 31 is picked out, running from 0° to 6° with its axis on 3°. Coordinates do not carry across a zone boundary — a point on either side of one has two entirely different eastings, and nothing in the numbers says which zone they belong to.

UTM and the zone system

Sixty separate maps of the world, each six degrees wide, each with a scale factor of 0.9996 chosen so the projection is wrong everywhere and less wrong at the edges. Every constant in the definition is a measured trade rather than a convention.

families · Ellipsoid
Molodensky against the exact route, NAD27. The distance on the ground between where each shortcut puts the transformed point and where the exact Cartesian route puts it, at 0 metres of ellipsoidal height, on a logarithmic scale. The full formulae stay within 0.8 centimetres across every latitude drawn; the abridged form, which replaces two ellipsoid-difference coefficients with one combined term, is worst at 40 centimetres — a factor of 48. The abridged curve dips near 45°, where the combined coefficient happens to equal the pair it replaces.

Molodensky's shortcut

The exact way to shift a datum needs an iteration nobody wanted to run on a 1950s machine, so a direct formula was derived instead. It lands within centimetres — and the abridged version everybody quotes is a different formula, worse by a factor of fifty.

datums · Datum
What treating the Earth as a sphere costs, per journey. The ellipsoidal geodesic minus the spherical great circle, in kilometres, for five journeys. The correction is a few tenths of a per cent and it changes sign: a route running east–west at mid latitude is longer on the ellipsoid, and one running along a meridian is shorter, because an oblate body is fatter round the equator and flatter pole to pole. Both distances are computed — Vincenty's iteration against the haversine formula — and the ellipsoidal one is checked against a geodesic obtained by integrating its own differential equation.

Geodesics on the ellipsoid, and why they are hard

The shortest path on a flattened Earth is not a plane curve, has no closed form, and can be longer or shorter than the spherical answer depending on which way it runs. Every practical method is a series or an iteration, and the correction changes sign.

paths · Paths
Five latitudes that are not the latitude, on WGS84. Each curve is the amount by which one auxiliary latitude falls below the geodetic latitude a coordinate actually means, in arcminutes. All five vanish at the equator and at the poles and peak near 45°, where the geocentric latitude is 11.55 arcminutes below the geodetic one — about 21.4 km on the ground. The curves never cross, which is forced by the algebra rather than by this ellipsoid's particular flattening.

Geodetic against geocentric latitude

Two angles, both called latitude, differing by eleven and a half arcminutes at their worst. One is what every coordinate means and the other is what every spherical formula assumes, and the gap between them is twenty-one kilometres on the ground.

wrong · Ellipsoid
Where the inverse problem stops converging, 3° around the antipode of London. Every target in a 6° square centred on the point diametrically opposite London, shaded by how many iterations Vincenty's inverse formula needed to find the geodesic to it. 3 of 625 — 0% — never converged at all, and the worst success took 113 steps against a handful anywhere else on Earth. The failure is not a defect in the formula: near the antipode the shortest path is nearly ambiguous, and an iteration looking for one answer is being asked which of many.

The route with no shortest path

Between a point and the point diametrically opposite there are infinitely many shortest routes and no shortest route, and the standard formula for the distance between two places stops converging in a neighbourhood of it. The failure is a property of the question rather than a defect in the answer.

paths · Paths
What changing the datum alone does to a coordinate. The distance on the ground between a point as read on its national datum and the same numbers read on WGS84, computed through the published seven-parameter transformation. The shifts run from 49 to 166 metres. For comparison, the scale error a UTM zone introduces at its edge is under a metre per kilometre — so the datum, which is usually left unstated, dominates the projection, which is usually argued about.

Datum shifts dwarf projection errors

The projection argument is conducted in parts per million and the datum question is answered in hundreds of metres. A coordinate whose datum is unstated is out by more than any projection choice could ever put it, and almost nobody checks.

wrong · Ellipsoid
The radius of equal curvature, from equator to pole. The radius of the sphere that has the same Gaussian curvature as the ellipsoid does, at each latitude. On WGS84 it runs from 6357 km at the equator to 6400 km at the pole — the polar region is the flattened part of a squashed ball and is therefore the LEAST curved — so K itself varies by 1.35%. A surface whose curvature varies cannot be laid on one whose curvature does not, so no map from the sphere to the ellipsoid is faithful either, and the least a conformal one can vary in scale is 6739 parts per million.

The curvature of the Earth is not one number

An ellipsoid's Gaussian curvature varies by 1.35% from equator to pole, so the sphere is not developable onto the ellipsoid any more than the plane is onto the sphere. The spherical approximation is a projection with an irreducible cost, and the cost is 6,739 parts per million.

impossibility · Curvature
Three surfaces, and the two heights between them. The ellipsoid, the geoid and the ground, with the two heights a coordinate can carry. Ellipsoidal height h is what a satellite fix returns and is measured from a surface defined by four constants. Orthometric height H is what a level and a staff measure and is referred to the geoid — the equipotential surface that best fits mean sea level. They differ by the separation N, drawn here as 45 metres because that is a stated input rather than a computed one: a geoid model is a data product with a truncation degree in it, and this site computes rather than downloads. The arithmetic h = H + N is exact whatever N is.

Height above what?

A satellite reports a height above a mathematical surface. A level and a staff report a height above the surface water settles on. The two disagree by tens of metres, both are correct, and only one of them decides which way a pipe drains.

datums · Height
"Within 0.01°" on the ground, at five latitudes. The same condition drawn at five latitudes, all at one scale. A degree of latitude is a fixed distance — 1106 metres here, varying by less than one per cent from equator to pole — while a degree of longitude collapses as cos φ, from 1113 metres to 289 at 75°. So the "circle" is an ellipse of 3.86:1 there, and it encloses 26 per cent of the ground the same condition covers on the equator. Even on the equator it is not round: M is smaller than N by the flattening, so the shape is 6694 parts per million shorter north–south than east–west.

A degree is not a unit of length

"Within 0.01 degrees" is a condition anybody can write and no instrument can measure. On the ground it is an ellipse — 1,106 metres north–south and 558 east–west at 60° — and even on the equator it is not a circle, because the meridian's radius of curvature is smaller than the parallel's by the flattening.

applied · Dataset

Equal-area on the wrong body

The site's headline is that Web Mercator puts geodetic latitudes into a spherical conformal formula and stops being conformal. The same sentence is true with "equal-area" in it and nobody says it: the areal factor is 1.00674 at the equator, 0.99332 at 88°, and averages to almost exactly one — so every check that adds up areas passes while every cell is wrong.

wrong · Audit

The ellipsoid is a level surface

WGS84 publishes two dozen constants and defines four of them. The other twenty are consequences — polar gravity, the potential of the ellipsoid, the coefficient that dominates the Earth's gravity field — and every one comes back here from a, f, GM and ω to the last digit published.

datums · Height

Where the shortest route stops being the only one

On a sphere there is exactly one point with no shortest route from a given place: the antipode. On the ellipsoid the Earth actually is, that point is an arc — sixty-six kilometres of the antipodal meridian for a point on the equator, half a kilometre for one at 85°, and every point of it reachable by two different geodesics of exactly equal length.

paths · Paths

Computing an area needs a surface

A shoelace over a ring of coordinates returns a number whatever the coordinates are. For one twenty-by-ten-degree cell it returns 1.75 times the true area in degrees, 3.06 in a conformal plane, and exactly the closed form in an equal-area one — and the closed form itself is 0.45 per cent out, because the sphere is a model too.

applied · Dataset

The normal section is not the geodesic

A theodolite at A sighted on B swings in one plane and the instrument at B sighted back swings in another, so the two observations trace different curves on the ground — 79 metres apart over 3,026 kilometres — and the shortest path is neither of them. The lengths differ by 2.4 millimetres, so the wrong curve measures the right distance along the wrong ground.

wrong · Ellipsoid

The shortest route is not at sea level

Ten rungs route on a surface and nothing is ever flown on one. The offset of a sphere is a sphere, so at altitude the great circle is the great circle. The offset of an ellipsoid is not an ellipsoid — its radii of curvature are M + h and N + h, which belong to no ellipsoid — so the shortest route at cruising height does not lie above the shortest route on the ground.

paths · Paths

The flattening is not a free parameter

An ellipsoid is usually presented as two numbers somebody fitted. One of them is not free — Clairaut's theorem relates the shape of a rotating body to the gravity on it, and the relation holds on WGS84 with a residual of 3.1×10⁻⁵ — which is 2.74 times f², exactly what a first-order theorem is entitled to.

datums · Ellipsoid

The figure of the Earth was measured

Two expeditions went to Lapland and Peru to measure the length of a degree of latitude, and the whole signal separating a flattened Earth from a round one is a kilometre in a hundred and eleven. Inverting two arcs amplifies their error by 118 — and a kilometre of error returns a lemon-shaped planet.

datums · Ellipsoid

A coordinate on another body

Mars publishes two latitudes for every point and they differ by up to 0.338°, which is twenty kilometres of ground — almost exactly the same distance as the Earth's own 0.192°, because Mars is smaller by nearly the same factor. The libraries that compute either have known both numbers since this collection's early essays and had never been pointed anywhere but here.

datums · Bodies

How big a triangle it takes

Gauss proved that a surface-dweller can read the curvature off a triangle's angles. Doing it is another matter: a fifty-kilometre triangle has 5.49 seconds of excess, a one-second theodolite gives that excess a standard deviation of 1.73, and reading K to one per cent needs a side of 281 kilometres. Not one of the great surveys built a triangle within a factor of three of that.

impossibility · Curvature

The equator is not a circle either

Eleven rungs price what pretending the Earth is a sphere costs, and every one of them replaces the sphere with a surface of revolution — a body whose equator is a circle. It is not. The two equatorial radii differ by seventy metres, the two surfaces part by thirty-five, and the auxiliary latitudes every ellipsoidal formula is written in stop existing.

wrong · Ellipsoid

The inverse of the series is not the series of the inverse

Transverse Mercator on an ellipsoid has no closed form, so every national grid computes it as a truncated series. There are two of them — one out and one back — and they are separate approximations. Composing them does not give the identity: at the order every grid formula in ordinary use is written to, a point three degrees from the central meridian comes back 0.39 mm north of where it started.

families · Ellipsoid

The same projection on a different body

A projection's distortion does not depend on the body's size at all — only on its flattening — so Web Mercator's 0.3848° of angular deformation is not a fact about Web Mercator. It is 2f radians, and the same mistake made on Mars measures 0.6765°, on Jupiter 7.68°, and on a body scaled ten times larger than Mars the identical number to twelve figures.

datums · Bodies

The bound on the body the country is on

The best conformal grid a country could have had was computed against a bound solved on a sphere, with a note saying the flattening was second order and unquantified. It is second order for a named projection — every family's best candidate moves by under 0.7 per cent — and it is 22 per cent for the bound, because an optimal map has already cancelled its own variation and has nothing left to hide a new one in.

practice · Grid

The datum hides inside the projection's parameters

Six essays recover a projection from control points with the body it was drawn on taken as known. It is not known, and the fit cannot find it: a 99-metre datum shift is absorbed to 99.7 per cent on a two-degree sheet and to 85 per cent on a hemisphere, leaving eleven parts per million of the map behind.

wrong · Identify

A rotation is not absorbed the way a shift is

The previous rung expected a datum's rotations to be the part a plane fit could not swallow. They are the part it swallows best — 93 times better than the translations over a hemisphere, and 24 times better per metre moved — and the reason is that the rotation which matters is a change of longitude, which is a symmetry of the map.

wrong · Identify

Which small quantity the series is in

Every ellipsoidal formula in this collection is a truncated series in the third flattening, inherited from Krüger in 1912 and justified nowhere. Measured against the classical expansion in e², at four sine terms it is 1,175 times more accurate — and at one and two terms it is fractionally worse, which is not what the folklore implies.

wrong · Ellipsoid

Four radii of the Earth

Ten rungs handle the ellipsoid with an auxiliary latitude. Every one of those constructions also needs a radius, and the radius that makes each property exact is a different number: the published 6371 km is right for an area to half a part per million and wrong for a meridian distance by 559 — while the radius a conformal map needs is not a constant at all, and spans 6,739.

families · Ellipsoid

A meridian boundary moves when its datum does

The 141st meridian is the one boundary description in this collection with no geometric ambiguity in it: every reading of it is the same curve, exactly. It has a different one. A longitude refers to a datum, the 1825 convention named none, and the line of longitude exactly 141° west sits 129.0 metres apart on NAD27 and WGS84 — a ribbon 1,039 kilometres long and 134 square kilometres in area.

practice · Boundary

An equidistance line belongs to a surface

A maritime boundary is very often defined as the line equidistant from two coasts — a description with no coordinate in it and no curve to choose between. It has a third ambiguity: equidistant measured how. On the ellipsoid, on the sphere, and with a ruler on three different charts, the same four basepoints give lines up to 39.6 kilometres apart and 2,816 square kilometres of seabed between them.

practice · Boundary

The radius of curvature is two numbers

Twelve essays work on the ellipsoid and every one of them takes a radius when it needs one. There are two at every point, they differ by 42.70 kilometres at the equator, and the single number every table prints is out by 5,583 parts per million on a line running north.

datums · Ellipsoid

The height a coordinate does not carry

Ten rungs treat a datum shift as a map from one pair of angles to another. It is not one: the transformation runs through Cartesian coordinates, so the answer depends on the height — by 7.7 to 121 millimetres per kilometre depending on the datum and the place, which puts Lhasa 432 millimetres from where a height of zero would have said.

datums · Datum

Named alongside it

The objects these essays reach for when they reach for this one.

FlatteningDatumVerificationToleranceRadius of curvatureGeodesicWGS84ConventionNumerical integrationConformalityAuxiliary latitudeGeodetic latitude

All concepts