Skip to content

Partial moments and CDF: mirror the NaN degree, 0/0 fill and survival fixes - #164

Closed
gitRasheed wants to merge 5 commits into
rasheed/api-polishfrom
rasheed/mirror-moments-cdf
Closed

gitRasheed wants to merge 5 commits into
rasheed/api-polishfrom
rasheed/mirror-moments-cdf

Conversation

@gitRasheed

@gitRasheed gitRasheed commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Three fixes from the R partial-moment PRs that the earlier mirrors did not have yet. A NaN degree slipped through the pure-Python degree check. The 0/0 fill replaced every NaN ratio at degree > 0, including overflow NaNs. The multivariate survival at degree > 0 mixed subset ratios with different normalisers and put mass where no observation lies above the target.

Stacked on #163; merge with a merge commit.

Changes

  • lpm, upm, the ratios, the co-moments and pm_matrix raise on a NaN degree in the pure path, as the native path already did. lpm_ratio(nan, 2, [1, 2, 3]) returned 0.6667.
  • The 0/0 fill checks once that the sample is constant and fills only targets equal to it, as R's fill_degenerate_ratio does. Other NaNs stay NaN: lpm_ratio(3, 0, [1e200, -1e200]) goes from 0.5 to NaN, as in R. The pure-path nns_cdf([2] * 20000, 1) goes from 54.7 s to 39.9 s.
  • At degree > 0, multivariate survival is the normalised Co.UPM_nD, also for d = 2. For the matrix B with target (4, 1, 1), d = 3 survival goes from 0.0824 to 0, and d = 2 from 0.1786 to 0. Degree-0 survival, hazard and cumulative hazard are unchanged.
  • docs/parity.md and sync/nns_source.json name the R PRs the cache comes from.

Tests

New parity cases for the overflow ratio and the degree-1 survival, and an invariant for the NaN degree in both paths. The three degree-1 trivariate NNS.CDF.custom entries predate the R fix and are refilled. From this slice up, the parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92.

Mirrors OVVO-Financial/NNS#73, OVVO-Financial/NNS#74 and OVVO-Financial/NNS#85

Update

This push mirrors OVVO-Financial/NNS#104: the degree-0 joint survival above 3 columns counts the strict upper orthant directly. On a 20x20 matrix, 12 of 20 hazards were wrong (max error 2.458) and the call took 17 s; it is now exact.

Review

GPT-6.1 Sol reviewed this slice and found nothing. It checked NaN degrees, constant samples, ties and multivariate survival against live R on both paths.

_as_degree checked `degree < 0`, which is False for NaN, so in the
pure-Python path a NaN degree went through: lpm returned NaN and
lpm_ratio filled that NaN with the degree-0 ratio, which looks valid.
R's nns_check_degree (native-kernels) stops on a negative or NaN degree.
Use `not degree >= 0` so lpm, upm, the ratios, the co-moments and
pm_matrix raise on NaN in both paths.

Before -> after (NNS_DISABLE_NATIVE=1):
  lpm(nan, 2, [1, 2, 3])         nan    -> ValueError (R: error)
  lpm_ratio(nan, 2, [1, 2, 3])   0.6667 -> ValueError (R: error)
The native path already raised.
…he R fix)

_fill_degenerate_ratio replaced every NaN ratio at degree > 0 with a
degree-0 partial moment over the whole sample. As in R's
fill_degenerate_ratio (pm-ratio-targets), check once that the sample is
constant and fill targets equal to it with 1 (lower) / 0 (upper). A
ratio is 0/0 only there; other NaNs, such as Inf/Inf from overflow, now
stay NaN as in R. The guard is `not degree > 0`, so a NaN degree never
reaches the fill.

Before -> after (= R):
  lpm_ratio(3, 0, [1e200, -1e200])          0.5 -> nan
  lpm_ratio(1, [2, 3, 4], [3, 3, 3, 3])     0 1 1 (unchanged)
  nns_cdf([2] * 20000, 1), pure path        54.7 s -> 39.9 s
New LPM.ratio and UPM.ratio entries for the overflow case.
…(mirror of the R fix)

At degree > 0 the inclusion-exclusion sum mixes subset ratios that each
have their own normaliser, so it is not a joint moment and gives
positive survival where no observation lies above the target. As in R
NNS.CDF (cdf-moments-var), keep inclusion-exclusion at degree 0 and use
the normalised Co.UPM_nD at degree > 0, computed as Co.LPM_nD(-X, -t).
This applies to d = 2 as well.

Before -> after (degree 1, = R):
  B = [[0,1,3],[1,0,2],[2,3,0],[4,2,1]]
    d = 3, target (4, 1, 1), survival    0.0824 -> 0
    d = 2, target (4, 1),    survival    0.1786 -> 0
  trivariate test matrix, target (0.1, 0, -0.2)
    survival                             0.0369 -> 0.0099
    cumulative hazard                    3.298  -> 4.617
Degree-0 survival, hazard and cumulative hazard are unchanged.

The three degree-1 trivariate NNS.CDF.custom entries predate the R fix
and were refilled from it (the three degree-0 ones came back identical);
two entries are added for the new tests.
The provenance note in docs/parity.md and the sync/nns_source.json notes still
pointed at the local fix/combined branch. The cache is generated from R 13.3
with the fixes in OVVO-Financial/NNS pull requests 73 to 92.
…ror of the R fix)

Inclusion-exclusion over the 2^d column subsets took 17 s at 20 columns and
its alternating sum cancelled to a cumulative hazard off by 2.46. As R now
does, above 3 columns the survival is the share of rows above t in every
column; up to 3 the inclusion-exclusion path stays.
@gitRasheed
gitRasheed force-pushed the rasheed/mirror-moments-cdf branch from 32224b6 to 6e93b21 Compare November 2, 2026 23:09
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head 6e93b21aeba2a419e216825c290f2082207b6186 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/mirror-moments-cdf branch November 7, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant