Skip to content

nns_diff stop rule and the stack ALL candidate: mirror the later R fixes - #168

Closed
gitRasheed wants to merge 6 commits into
rasheed/mirror-dominancefrom
rasheed/mirror-regression-stack
Closed

gitRasheed wants to merge 6 commits into
rasheed/mirror-dominancefrom
rasheed/mirror-regression-stack

Conversation

@gitRasheed

@gitRasheed gitRasheed commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

The nns_diff bracket stop used tol * max(1, |B|), and |B| grows with any constant added to f, so an offset loosened the stop and changed the derivative. The stack's ALL candidate ran a full nns_reg(n_best="all") in every fold, rebuilding partitions the fold already had.

Stacked on #167; merge with a merge commit.

Changes

  • nns_diff stops at tol * max(|point|, |f(x) - B|), a derivative error of about tol * max(1, |f'(x)|), and the docs say what tol controls. sin(x) + 1000 at 1 with tol 1e-4 goes from 0.5623305 after 1 iteration to 0.5402970 after 8, as for sin(x).
  • The multivariate ALL candidate is predicted from the fold RPM with k = nrow(RPM), the same estimator. Output is bit-identical in 11 configurations, native and pure, and a 3-predictor Method 1 fit at n = 2000 goes from 2.18 s to 1.83 s.
  • The stack docstrings, conventions.md and api_status.md no longer call sorted character levels a divergence, since R's NNS.stack now sorts too. The anova.py comment saying R's two-sample path propagates NaN goes, since R now drops NaN there.

Tests

Six of the eleven NNS.diff.custom entries predate the R fix and are refilled; iterations move by one or two and derivatives in the 10th digit. The R helper gains a sin_offset function and an optional tol. The parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92. The exp derivative case now uses libm exp, as R does: with the new stop its inferred h is about 6e-9, and an ulp of difference in NumPy's SIMD exp moved it to 4.1e-9 on the CI runners.

Mirrors OVVO-Financial/NNS#80, OVVO-Financial/NNS#81 and OVVO-Financial/NNS#83

Update

This push mirrors OVVO-Financial/NNS#103. For a two-valued numeric response in regression mode, a custom obj_fn scores the final stack on the response values. It used to see the class codes 1 and 2 mixed with the values, {-5, 1, 2, 5}; it now sees {-5, 5}.

Review

GPT-6.1 Sol reviewed this slice and found nothing. The 48 old/new stack cases were bit-for-bit identical on both paths.

…of the R fix)

The bracket stop used tol * max(1, |B|). The derivative is
(f(x) - B) / point, so its error is the bracket width divided by |point|,
and |B| grows with any constant added to f: an offset loosened the stop
and changed the derivative. As in R NNS.diff (dydx-diff), use
tol * max(|point|, |f(x) - B|), a derivative error of about
tol * max(1, |f'(x)|), and say what tol controls.

Before -> after (= R):
  sin(x) + 1000 at 1, tol = 1e-4   0.5623305 (1 iteration)
                                   -> 0.5402970 (8 iterations, as sin(x))
  x^3 at 2, default tol            12.000000043524 (32) -> 12.000000043817 (31)
Six of the eleven NNS.diff.custom entries predate the R fix and were
refilled (iterations move by one or two, derivatives in the 10th digit);
the other five came back identical. The R helper gains a sin_offset
function and an optional tol, which only enters the payload when given.
… of the R fix)

Scoring ALL with a full nns_reg(n_best="all") call in every fold rebuilt
the partitions the fold had already built. As in R NNS.stack
(stack-boost), predict the validation rows from the fold RPM with
k = nrow(RPM): the same estimator, so the output does not change.

Method 1, 3 predictors, y = a + b^2 - c + noise, n = 2000, native:
  2.18 s -> 1.83 s
Outputs are identical (native and NNS_DISABLE_NATIVE=1) for dist L1, L2,
NNS and FACTOR with order None and "max", method (1, 2), a balanced
3-class response and a pure-noise fit.
R's NNS.stack now codes character responses and predictors in sorted
level order, as NNS.boost does, so the stack module docstring, the
_prepare_inputs docstring, conventions.md and api_status.md no longer
call it a divergence. R's two-sample NNS.ANOVA now drops NaN like the
multi-group path, so the anova.py comment saying it propagates NaN goes.
The derivative-scaled B-bracket stop moves the quintic case from Inferred h
1.187e-09 after 31 iterations to 3.478e-09 after 33, in R and Python alike.
With the derivative-scaled stop the inferred h for exp at 0.5 is about 6e-9,
so an ulp of difference in np.exp's SIMD loop moved it to 4.1e-9 on the CI
runners. R uses libm exp, as the tanh case already does.
nns_stack codes a two-valued numeric response as classes 1/2, so a custom
obj_fn saw the response values during learner selection and the codes in the
final stack ({-5, 1, 2, 5} for a +-5 response). As R now does, the final
stack's objective decodes predicted and actual codes into the two sorted
response values first.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head bd3e248dfee367a3e9ab585dc1692c2209c6f942 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/mirror-regression-stack branch November 7, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant