Flying a curve in straight legs

The great circle is the shortest route and it has a property that disqualifies it as an instruction: the heading changes at every moment along it.

The instruction problem #

Mercator exists because a constant compass bearing is a straight line on it, and a constant compass bearing is the only instruction a helmsman or an autopilot can follow without continuous computation. That was the founding trade of the subject: the rhumb line is steerable and the great circle is short.

The resolution everyone actually uses is neither. Pick a few points on the great circle, call them waypoints, and fly a constant heading between consecutive ones. Each leg is a rhumb line — that is what a constant heading is — and the whole route is a polygon of rhumb lines inscribed in the great circle.

That is composite sailing, and it is what a flight plan is. What it costs is two numbers, and both are computable.

The two costs #

How far from the intended route the flown path strays. Each leg leaves the great circle immediately after its waypoint, bulges to one side, and returns at the next. The maximum of that departure — the cross-track error — is what matters for airspace, for terrain and for anyone else in the vicinity.

How much further it is. The shortest route is not straight on the page, and the flown one is not shortest on the ground. The polygon is longer than the arc it is inscribed in, because a rhumb line between two points is longer than the great circle between them, and the excess accumulates over the legs.

Measured on London to Tokyo, a 9,559 km great circle:

The one-leg row is the pure rhumb line, and its 18.17% excess is the number the trade-off essay is built around: the price of being able to steer at all, before any composite scheme is applied.

The law is quadratic #

The exponent is the reason a small number of legs works so well, and it is worth deriving rather than fitting — with care, because the obvious derivation is for the wrong pair of curves.

A leg is not a chord of the great circle. It is a rhumb line between two points on it, and a rhumb line and a great-circle arc with the same endpoints differ because the great circle’s bearing swings while the rhumb line’s does not. Over a leg spanning $\Delta\lambda$ of longitude at mean latitude $\varphi$, the great circle’s bearing changes by roughly $\Delta\lambda\sin\varphi$, and the separation between the two curves is of order one eighth of the leg length times that bearing change.

Both factors are proportional to $1/n$, so the departure falls as $1/n^2$ — quadratic, as the sagitta argument would also have said, but for a different reason and with a different coefficient. The coefficient carries $\sin\varphi$, which is why the law is a statement about where on the globe the route runs and not only about how long it is.

The fitted exponent comes out at −1.88 over the whole range drawn, short of −2 because the one- and two-leg cases are far outside the small-angle regime. Restricted to eight legs and above it is −1.98.

The check on the derivation is the total bearing swing. London to Tokyo departs on 31.7° and arrives on 156.3°, a swing of 124.6°; the formula $\Delta\lambda\sin\varphi$ with $\Delta\lambda = 139.8°$ and a mean latitude of 60° predicts 121°. Agreement to three per cent on a quantity derived from the small-angle expansion of a different curve is about what the expansion is worth, and it is the reason to trust the exponent rather than the constant.

Two things follow that a navigator can use.

The first few legs are worth far more than the rest. Going from one leg to four removes 90% of the departure. Going from sixteen to sixty-four removes another 94% of what is left, which by then is 20 km, and nobody cares.

A tolerance sets the leg count directly. For a departure of at most $d$ over a route of length $L$ on a sphere of radius $R$, the requirement is $n \gtrsim \Theta\sqrt{R/8d}$, which for London to Tokyo and $d = 10$ km gives 13.7 — so fourteen legs, and the measurement says sixteen is the first power of two that suffices.

Every leg bulges the same way #

The flown route never crosses the great circle between waypoints. It leaves on one side, returns, and leaves again on the same side, so the polygon lies entirely inside the arc.

That is asserted on every build and it is a stronger check than it looks. A rhumb line between two points on a great circle is on the equator side of it, always, because a rhumb line holds its bearing while a great circle’s bearing swings towards the pole and back — the vertex is exactly the place where the swing turns round. An implementation that generated its legs by interpolating latitude and longitude linearly, rather than by integrating the constant-bearing condition, would produce a path that crosses the arc, and the sign check catches it.

This site has made that error and recorded it: rhumbPath interpolated linearly in its first version, produced a curve that was not a rhumb line, and bowed by nearly one per cent of its length on Mercator, where a rhumb line must be dead straight.

What the headings do #

The list of headings for a plan is where the underlying geometry becomes visible.

For London to Tokyo the three-leg plan flies 50°, 109°, 149°: north-east, then east, then south-east. The sixteen-leg plan runs 34°, 38°, 45°, 53°, 64°, 77°, 92°, 107°, 120°, 130°, 138°, 143°, 148°, 151°, 153°, 155°.

Two features of that sequence are the great circle’s own properties, sampled.

The total swing is 121° and it is the same for every plan, because the first heading tends to the great circle’s initial bearing and the last to its final bearing as the legs get shorter. Those are properties of the route, not of the plan.

The swing is fastest in the middle, where the sequence steps 64°, 77°, 92°, 107° in four legs while the ends step by two or three degrees each. That is the route passing its vertex — the highest latitude it reaches, where the track is running due east and the bearing is changing at its greatest rate.

So the leg list is a coarse record of the same quantity Clairaut’s relation gives in closed form, and comparing the two is the natural check on a plan.

Where the tolerance comes from #

Ten kilometres is the figure used above and it is chosen to be legible rather than because anyone flies to it. The tolerances that actually apply are set by separation standards and by the width of the airway, and both are much tighter.

The relevant modern standard is Required Navigation Performance, which is quoted as a lateral containment in nautical miles — RNP 10 means the aircraft is to remain within 10 nautical miles of the centreline 95% of the time, RNP 4 within four, RNP 1 within one. Those figures include every source of error, of which the leg approximation is only one, so the geometric budget is a fraction of them.

Reading the measurement against RNP 4, or 7.4 km: London to Tokyo needs at least 18 legs on the geometry alone. That is the sense in which the number of waypoints on an oceanic track is not arbitrary — it is the leg count at which the geometric departure disappears into the error budget, and the reason the tracks are quoted at ten-degree intervals of longitude is that ten degrees is about the right leg length at these latitudes.

The shorter the route, the fewer the legs #

The whole calculation scales with the length of the route, and it scales quadratically there too.

New York to Madrid is 5,768 km against London to Tokyo’s 9,559, and its departures run

a factor of between four and seven better at every leg count. The ratio of the route lengths is only 1.66, so length is not the whole story — the rest is the bearing swing, which is 125° on the transpacific route and 48° on the transatlantic one, because the second crosses half the longitude at a lower latitude.

That is the useful generalisation. A route’s difficulty for a composite plan is set by how much its bearing changes, not by how far it goes. An east–west route at high latitude is the hard case and a north–south route is the easy one: a meridian is simultaneously a great circle and a rhumb line, so a route due north needs one leg and no waypoints at all, at any length.

The other composite, and why it is the same construction #

Composite sailing in its original sense solved a different constraint with the same trick, and putting the two side by side is the clearest way to see what a waypoint is for.

A ship crossing the Southern Ocean on a great circle would be carried far further south than it could survive. The eighteenth-century answer was to fly the great circle until it reached a chosen limiting latitude, run along that parallel, and rejoin a great circle for the remainder — a route made of three pieces, each of which is steerable, and whose junctions are chosen so that a constraint is respected.

That is structurally identical to the modern plan. In both cases the geodesic is unusable as an instruction and is replaced by a chain of curves that are; the difference is only which family of curves the pieces are drawn from and what the junctions are chosen to satisfy. The limiting-latitude version even has the vertex as its explicit subject, since the limiting latitude is exactly a cap imposed on the vertex.

The generalisation worth carrying is that a route is a compromise between two properties of a curve: being short and being followable. A projection is a compromise between two properties of a map. Every projection minimises something and so does every route, and in both cases naming the objective explains more than naming the shape.

What a modern autopilot actually does #

The technology removed the constraint and the structure survived, which is worth saying plainly because it looks like an anachronism.

An inertial or satellite navigation system can compute a great-circle track continuously and steer it exactly, with no legs at all. What it cannot do is file it. A flight plan is a contract with air traffic control, expressed in points that both parties can name, and a list of coordinates is nameable in a way that “the geodesic from here to there” is not — particularly when the two parties are using different geodetic datums, at which point the difference between the datums is tens or hundreds of metres and the difference between two descriptions of a curve is unbounded.

So the leg structure is now a communication protocol rather than a steering technique, and the tolerances above are the tolerance of the protocol: not how accurately an aircraft can hold a track, but how accurately the track it holds matches the one everybody else believes it is on.

On Mercator each leg is what it is on the ground: a straight line held at a constant bearing.

What was computed here #

The waypoints are placed at equal fractions of the great circle by spherical interpolation. Each leg is a rhumb line generated by integrating the constant-bearing condition — $d\varphi = \delta\cos\theta$, $d\lambda = \delta\sin\theta/\cos\varphi$ — with a midpoint rule, so that no part of the calculation uses Mercator, which is the projection whose behaviour is being asserted elsewhere.

The cross-track error of each sampled point is the spherical formula $R\arcsin(\sin d_{13}\sin(\theta_{13} - \theta_{12}))$, signed, and the maximum over 24 samples per leg is reported. Three claims are asserted: the departure falls monotonically with the leg count, the fitted exponent is within 0.15 of −2, and every sample of a one-leg route lies on the same side of the great circle.

What the pictures cannot show #

The two lines converge visually long before they converge numerically. At eight legs the drawing shows one curve and the measurement shows 77 km, which is a real quantity in an airspace whose lateral separation might be 90 km.

The maps also draw the flown route on a projection, so the apparent gap between plan and curve is partly the projection’s. That is why the departures are measured on the sphere and quoted in kilometres rather than read off the page.

The same economics, for a line in a file #

The leg count this essay computes has an exact counterpart in stored geometry, and the arithmetic is identical because the geometry is.

Densifying a line — cutting it into enough straight pieces that no piece strays further than a stated tolerance from the ground — obeys the same quadratic law as a leg approximation, so the vertex count goes as the reciprocal square root of the tolerance. On a 4,605-kilometre line: 8 pieces at 10 kilometres, 32 at one, 128 at a hundred metres, 512 at ten. A hundredfold tighter line for sixteen times the vertices.

How many legs a separation standard actually buys #

The quadratic law and one measured point fix the whole curve, so the leg count a given airspace requires can be read off rather than tabulated.

Eight legs give 77 kilometres of departure on this route, and the departure falls as the square of the count, so the product of the departure and the square of the legs is about 4,900 kilometre-legs² for London to Tokyo. Inverting it against the lateral standards actually in force:

The traditional handful of waypoints is exactly the fifty-mile standard, which is the separation the North Atlantic and Pacific tracks were built around and the one the leg structure grew up under. Tightening to RNP-10 roughly doubles the count and RNP-4 roughly triples it, which is the direction modern published tracks have gone — more waypoints, at closer spacing, on the same routes.

Two things follow that the law makes obvious and a table of practice does not.

The cost of a tighter standard is a square root, so it is affordable. Halving the tolerance costs 1.41 times the waypoints, not twice. Going from fifty nautical miles to four — a factor of twelve and a half in tolerance — costs a factor of 3.5 in legs, which is why the transition happened without anybody rewriting the protocol.

And the constant is route-specific. The 4,900 belongs to a 9,559-kilometre great circle at these latitudes; a shorter route needs fewer legs for the same tolerance, in proportion to its own length, because the departure of a single leg goes as the square of the leg and the leg is the route divided by the count. The leg count for a stated tolerance is therefore proportional to the route’s length, which is the rule a flight planner uses without deriving it.

Who found it, and when #

Composite sailing is in the navigation manuals from the eighteenth century, usually as the combination of great-circle and rhumb sailing designed to keep a ship out of latitudes it could not survive — a great circle whose highest latitude is capped by replacing its polar section with a run along a parallel.

The modern version is the same construction driven by a different constraint: not the ice, but the impossibility of holding a continuously varying heading. Inertial and satellite navigation removed that constraint decades ago and the waypoint structure survived, because air traffic control needs a route that can be stated — and a list of points is statable in a way that “the geodesic” is not.

Where this goes next #

Both ends of the paths ladder now have a fourth rung: the case where the shortest route is not unique, and the case where it cannot be followed. The remaining questions are about the maps rather than the routes, and the next one is what makes a family of projections a family at all.