Skip to content

geometry: use Newton's method in karney_inverse without the reduced length - #54

Open
pulk17 wants to merge 1 commit into
ClickHouse:clickhouse-1.90.0from
pulk17:karney-inverse-newton
Open

pulk17 wants to merge 1 commit into
ClickHouse:clickhouse-1.90.0from
pulk17:karney-inverse-newton

Conversation

@pulk17

@pulk17 pulk17 commented Oct 27, 2026

Copy link
Copy Markdown

lambda12 (boost/geometry/formulas/karney_inverse.hpp) takes the derivative of lam12 from meridian_length, but meridian_length computes the reduced length only when EnableReducedLength is set. strategy::karney uses a distance-only instantiation for length, so the derivative was always zero. Newton's method never ran, and every iteration fell back to bisection.

The first bisection step from the bracket [0, 180] is azimuth 90. When beta2 equals -beta1 up to rounding, lambda12 then takes the square root of a slightly negative number for cos_alpha2. The distance comes out NaN, and normalize_unit_vector asserts h > 0. Example on WGS84: from (0, -45) to (1e-10, 45), the distance should be 9969888.755955 m but was NaN, in both directions. This showed up in ClickHouse's lengthSpheroid: ClickHouse/ClickHouse#127764

The fix always requests the reduced length for the derivative, as GeographicLib does.

  • NaN results: a scan of 9.6 million segments between latitudes near -45 and 45 found 186902 NaN results before the fix and none after.
  • Asserts and UBSan: neither reports anything on 400 thousand degenerate and random segments.
  • Accuracy: results agree with an independent solver to the micrometer. Before, the bisection was off by up to 1.3 mm near the equator.
  • Speed: length is about 4 times faster on those segments.

…ength

lambda12 gets the derivative of lam12 from meridian_length, which computes
the reduced length only when EnableReducedLength is set. A distance-only
instantiation, like the one strategy::karney uses for length, therefore got
a zero derivative and fell back to bisection on every iteration. Starting
from the bracket [0, 180], the first bisection step is the azimuth 90, and
when beta2 is -beta1 up to rounding, lambda12 takes the square root of a
slightly negative number for cos_alpha2. The distance comes out NaN, and
normalize_unit_vector asserts.

For example, from (0, -45) to (1e-10, 45) on WGS84 the distance was NaN
instead of 9969888.755955 m. A scan of 9.6 million segments between
latitudes near -45 and 45 found 186902 NaN results before, none after.

Always request the reduced length for the derivative, like GeographicLib
does. Results now agree with an independent solver to the micrometer
(the bisection was off by up to 1.3 mm near the equator), and length is
about 4 times faster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant