Concept

Spherical approximation — where it appears

Treating a body as a sphere when it is an ellipsoid, which is exact for nothing and adequate for a great deal. Its error follows the flattening: about a third of a per cent for the Earth and about twice that for Mars, which is why the same substitution fails there.

Named by 12 essays across 5 fields — each of them below, with the objects they name alongside it.

Fitting a sphere to 10° of latitude at 54.5°. The Earth's Gaussian radius of curvature across the band, with two candidate spheres drawn against it. The global mean radius is 6371.0 kilometres and sits 2203 parts per million away from the ground here — 2.2 metres in every kilometre measured. The best local radius is 6385.0 kilometres and has no bias at all by construction, leaving 325 parts per million of spread that no sphere can remove, because the curvature varies across the band and a sphere's does not. The gain is a factor of 6.9.

A datum is fitted to a region

Forty countries adopted forty ellipsoids, and the usual explanation is that measurement was poor. It was not — a figure fitted to one country beats the global one over that country by a factor of seven, and what it removes is a systematic bias of two metres in every kilometre.

datums · Datum
The radius of equal curvature, from equator to pole. The radius of the sphere that has the same Gaussian curvature as the ellipsoid does, at each latitude. On WGS84 it runs from 6357 km at the equator to 6400 km at the pole — the polar region is the flattened part of a squashed ball and is therefore the LEAST curved — so K itself varies by 1.35%. A surface whose curvature varies cannot be laid on one whose curvature does not, so no map from the sphere to the ellipsoid is faithful either, and the least a conformal one can vary in scale is 6739 parts per million.

The curvature of the Earth is not one number

An ellipsoid's Gaussian curvature varies by 1.35% from equator to pole, so the sphere is not developable onto the ellipsoid any more than the plane is onto the sphere. The spherical approximation is a projection with an irreducible cost, and the cost is 6,739 parts per million.

impossibility · Curvature
How much curvature varies from place to place, body by body. The Gaussian curvature along a meridian, each body against its own smallest value, so the curves are comparable. Written from the bodies actually drawn: Earth varies by 1.0135, Mars varies by 1.0239, Vesta varies by 2.71, Phobos varies by 4.16, from the flattest place on each to the sharpest. A surface of constant curvature is a sphere and nothing else is, so the impossibility argument this field is built on has a local version on every real body: how flat a patch is depends on where the patch is.

Curvature that varies from place to place

The impossibility this site is built on was argued on a sphere, where the curvature is one number. On the Earth it varies by 1.35 per cent, on Jupiter by 31, on Vesta by a factor of 2.7 — and on a body with three axes it varies along a parallel, which no formula in latitude can express.

impossibility · Curvature
Four ways to build the same equal-area projection. The cylindrical equal-area projection's own areal scale factor, measured from its Jacobian against the metric of the body it is drawn for. On a sphere with the spherical formula it is one everywhere, which is the control. Feed the same formula the geodetic latitude a coordinate actually carries and measure against the ellipsoid it refers to, and it is 1.00674 on the equator and 0.99332 at 88° — a spread of 1.34 per cent, on a projection whose entire purpose is that there is no spread. Rescaling to make the totals agree does not repair it. The authalic northing q/2 does, exactly.

Equal-area on the wrong body

The site's headline is that Web Mercator puts geodetic latitudes into a spherical conformal formula and stops being conformal. The same sentence is true with "equal-area" in it and nobody says it: the areal factor is 1.00674 at the equator, 0.99332 at 88°, and averages to almost exactly one — so every check that adds up areas passes while every cell is wrong.

wrong · Audit
Three curves between 45°N and 55°N, 3026 km apart. The normal section observed from the first point, the normal section observed from the second, and the geodesic, each plotted as its distance to one side of the great-circle chord between the two ends. The two sections are 79.0 metres apart at their widest and the geodesic runs between them, 53.3 metres from the first. In LENGTH they differ by almost nothing — 2.40 millimetres over 3026 kilometres — so an observation that follows the wrong one measures the right distance along the wrong ground. On a sphere all three coincide exactly.

The normal section is not the geodesic

A theodolite at A sighted on B swings in one plane and the instrument at B sighted back swings in another, so the two observations trace different curves on the ground — 79 metres apart over 3,026 kilometres — and the shortest path is neither of them. The lengths differ by 2.4 millimetres, so the wrong curve measures the right distance along the wrong ground.

wrong · Ellipsoid
The bound was spherical, and the country is not. Three numbers per region, all in parts per million of scale spread. The first is the Chebyshev bound computed on the sphere. The second is that same optimal map used on the ellipsoid, which is what adopting it would actually deliver. The third is the bound with the ellipsoid-to-sphere factor put into the boundary condition, which is the real optimum. The penalty for using the spherical answer reaches 1.22 times — while a named candidate barely moves, because an optimal map has cancelled its own variation and has nothing left to hide a new one in.

The bound on the body the country is on

The best conformal grid a country could have had was computed against a bound solved on a sphere, with a note saying the flattening was second order and unquantified. It is second order for a named projection — every family's best candidate moves by under 0.7 per cent — and it is 22 per cent for the bound, because an optimal map has already cancelled its own variation and has nothing left to hide a new one in.

practice · Grid
Four radii of the Earth, and one that is a range. The four constants called the mean radius of the Earth, on a scale of kilometres, with the range of the local Gaussian radius √(MN) drawn behind them. Three of the four agree to about a part per million; the rectifying radius is 3560 metres smaller, which is 559 parts per million. The Gaussian radius spans twelve times that range on its own, which is why there is no such thing as the conformal sphere.

Four radii of the Earth

The ellipsoid is usually handled with an auxiliary latitude. Every such construction also needs a radius, and the radius that makes each property exact is a different number: the published 6371 km is right for an area to half a part per million and wrong for a meridian distance by 559 — while the radius a conformal map needs is not a constant at all, and spans 6,739.

families · Ellipsoid
What a geoid model leaves out, against the degree it stops at. The RMS of everything above the model's highest degree, from Kaula's rule — the statement that the normalised coefficients at degree n are about 10⁻⁵/n². The line is R × 10⁻⁵ ÷ n, so a model to degree 360 omits 17.7 centimetres and one to 2190 omits 2.9. Every orthometric height derived from such a model carries that as an error, and it is not quoted with the height.

The geoid model stops at a degree

Eleven essays treat the geoid as a surface that exists. Every geoid anybody uses is a series truncated at a degree, so every orthometric height derived from one carries an omission error nobody quotes with the height — eighteen centimetres at degree 360 — and the same truncation removes two thirds of the slope, which does not converge at all.

datums · Height
Four spheres are wrong about a direction before anything has moved. At 45° of latitude, the worst error over twenty-four starting directions of the azimuth each substituted sphere gives for a geodesic on WGS84, against the distance to its end. Geodetic latitude is wrong by 347″ at one kilometre, and the parametric, authalic and rectifying latitudes by 174″, 116″ and 86.8″ — a half, a third and a quarter of it — flat until a few hundred kilometres. The conformal latitude is right at the start and wrong by 54.3 mas at one kilometre, 5.43″ at 100 and 54.3″ at 1,000; the geocentric latitude starts at 620 mas and joins it. Bessel's sphere, which also corrects the longitude, is exact at every distance to the arithmetic's floor of 1.2e-7″.

A bearing on a sphere is decided by its latitude, not its radius

A sphere that stands in for the ellipsoid cannot be wrong about a direction by being the wrong size, because enlarging a sphere moves no angle. What it can be wrong by is the latitude put on it. Geodetic latitude turns every bearing at 45° by up to 347 arcseconds before anything has moved; the conformal latitude starts exact and drifts 54 arcseconds in a thousand kilometres; and only Bessel's sphere, which changes the longitude too, is exact at every distance.

wrong · Ellipsoid
Where the correction overtakes the sphere it is correcting for. The worst angle error of four substituted spheres against the size of the triangle, with Legendre's spherical excess on the same axes. The two that are wrong at first order in the flattening are flat in the side, because the error belongs to the latitude and not to the distance. The two that are right at first order have no such floor, so what is left is a term rising in proportion to the side. The excess rises as its square, so each pair meets exactly once, and the meeting decides which of the two corrections a computation is entitled to ignore: the geodetic latitude at 490 km, the rectifying latitude at 247 km, the conformal latitude at 23.8 km, the geocentric latitude at 17.6 km.

The correction is the smaller of the two corrections

Legendre's theorem subtracts a third of the spherical excess from each angle of a triangle and solves the rest as a plane figure. It is the correction every nineteenth-century computer applied and the one every textbook explains. Beside the error the choice of substituted sphere leaves in the same triangle it is small: on the thirty-kilometre triangle a first-order chain is made of, omitting it costs 166 millimetres in a computed side and using the geodetic-latitude sphere costs 43 metres. The two terms scale differently, so each sphere has a size at which the correction overtakes it — and for the two spheres a careful computer would have chosen, that size falls inside a real chain.

wrong · Ellipsoid
Along a meridian a good sphere's error adds up triangle by triangle; a bad base does not. A strip of thirty-kilometre equilateral triangles running north from 45°, each side computed from the last by Legendre's theorem on a substituted sphere. The scale error of the side just computed, against how many triangles lie between it and the measured base. The conformal sphere adds 7.90 ppm a triangle and reaches 158.1 ppm after 20; the geocentric, 158.9 ppm; the authalic, −52.4 ppm; the rectifying, 0.29 ppm. Shaded: three standard deviations of the scale error one-second angles leave, 3.96 ppm a triangle, growing as the square root of the count. Dashed: a base measured 8 ppm long, which stays 8 ppm at every triangle.

A strip of triangles carries the sphere's error to the next base

One triangle cannot tell a substituted sphere from a bad base: both leave the computed side a few parts per million long. A triangulation is not one triangle but a strip, each computed side the next one's base, and along it the two come apart completely. A bad base is one factor, the same at every triangle. A sphere's error is made afresh in each triangle and the strip adds it up, and what it adds up to at the far end is exactly the change in that sphere's own meridian scale between the two ends — 158 parts per million for the conformal sphere over twenty triangles north from 45°, predicted to half a part. Along a parallel it is nothing; at a check base three triangles on it is already three times the noise of one-second angles.

wrong · Ellipsoid
Adjusted to close on the second base, a sphere's drift of hundreds of ppm becomes a bow of under one. A strip of 40 thirty-kilometre equilateral triangles north from 45°, computed on the conformal sphere, between a base at each end. Unadjusted (dashed), the side error climbs to 315.0 ppm at the second base. Adjusted to close there, each triangle's share of the misclosure equal, what is left (solid) is the sphere's stretch less a straight line between the bases: 0.63 ppm at its largest, at triangle 23. Shaded: three standard deviations of what one-second angles leave in the adjusted strip, 37.6 ppm at the middle and nothing at either base.

An adjustment hides the sphere only in a strip of equal triangles

A strip computed on a substituted sphere drifts by the sphere's own stretch, 315 parts per million over forty triangles north from 45°, and an adjustment that closes it on a second base takes almost all of that away: in a strip of equal triangles what is left is the stretch less a straight line, 0.63 parts per million, a third-order bow sixty times below the noise of the old angles. A real chain is not made of equal triangles. The adjustment spreads the misclosure by how uncertain each triangle's angles are, the sphere put it in by how far north each triangle reaches, and the difference is a first-order residual that never clears the noise at any one side and that the old network's own geometry predicts exactly — so a re-observation matched against it reads the sphere at seven standard deviations.

wrong · Ellipsoid

Named alongside it

The objects these essays reach for when they reach for this one.

Auxiliary latitudeEllipsoidConformal latitudeFlatteningVerificationRadius of curvatureGeodetic latitudeScale factorClosed formGeodesicSurvey networkTolerance

All concepts