Repository navigation
nns_diff stop rule and the stack ALL candidate: mirror the later R fixes - #168
Closed
gitRasheed wants to merge 6 commits into
Closed
gitRasheed wants to merge 6 commits into
gitRasheed wants to merge 6 commits into
Conversation
This was referenced Nov 2, 2026
gitRasheed
force-pushed
the
rasheed/mirror-dominance
branch
from
November 2, 2026 04:12
b484a71 to
795254e
Compare
gitRasheed
force-pushed
the
rasheed/mirror-regression-stack
branch
from
November 2, 2026 04:12
a324683 to
febf707
Compare
…of the R fix)
The bracket stop used tol * max(1, |B|). The derivative is
(f(x) - B) / point, so its error is the bracket width divided by |point|,
and |B| grows with any constant added to f: an offset loosened the stop
and changed the derivative. As in R NNS.diff (dydx-diff), use
tol * max(|point|, |f(x) - B|), a derivative error of about
tol * max(1, |f'(x)|), and say what tol controls.
Before -> after (= R):
sin(x) + 1000 at 1, tol = 1e-4 0.5623305 (1 iteration)
-> 0.5402970 (8 iterations, as sin(x))
x^3 at 2, default tol 12.000000043524 (32) -> 12.000000043817 (31)
Six of the eleven NNS.diff.custom entries predate the R fix and were
refilled (iterations move by one or two, derivatives in the 10th digit);
the other five came back identical. The R helper gains a sin_offset
function and an optional tol, which only enters the payload when given.
… of the R fix) Scoring ALL with a full nns_reg(n_best="all") call in every fold rebuilt the partitions the fold had already built. As in R NNS.stack (stack-boost), predict the validation rows from the fold RPM with k = nrow(RPM): the same estimator, so the output does not change. Method 1, 3 predictors, y = a + b^2 - c + noise, n = 2000, native: 2.18 s -> 1.83 s Outputs are identical (native and NNS_DISABLE_NATIVE=1) for dist L1, L2, NNS and FACTOR with order None and "max", method (1, 2), a balanced 3-class response and a pure-noise fit.
R's NNS.stack now codes character responses and predictors in sorted level order, as NNS.boost does, so the stack module docstring, the _prepare_inputs docstring, conventions.md and api_status.md no longer call it a divergence. R's two-sample NNS.ANOVA now drops NaN like the multi-group path, so the anova.py comment saying it propagates NaN goes.
The derivative-scaled B-bracket stop moves the quintic case from Inferred h 1.187e-09 after 31 iterations to 3.478e-09 after 33, in R and Python alike.
With the derivative-scaled stop the inferred h for exp at 0.5 is about 6e-9, so an ulp of difference in np.exp's SIMD loop moved it to 4.1e-9 on the CI runners. R uses libm exp, as the tanh case already does.
nns_stack codes a two-valued numeric response as classes 1/2, so a custom
obj_fn saw the response values during learner selection and the codes in the
final stack ({-5, 1, 2, 5} for a +-5 response). As R now does, the final
stack's objective decodes predicted and actual codes into the two sorted
response values first.
gitRasheed
force-pushed
the
rasheed/mirror-regression-stack
branch
from
November 2, 2026 23:09
febf707 to
bd3e248
Compare
Contributor
|
Incorporated through NNS R 14.0 integration #192, merged as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
nns_diffbracket stop used tol * max(1, |B|), and |B| grows with any constant added to f, so an offset loosened the stop and changed the derivative. The stack's ALL candidate ran a fullnns_reg(n_best="all")in every fold, rebuilding partitions the fold already had.Stacked on #167; merge with a merge commit.
Changes
nns_diffstops at tol * max(|point|, |f(x) - B|), a derivative error of about tol * max(1, |f'(x)|), and the docs say what tol controls. sin(x) + 1000 at 1 with tol 1e-4 goes from 0.5623305 after 1 iteration to 0.5402970 after 8, as for sin(x).NNS.stacknow sorts too. The anova.py comment saying R's two-sample path propagates NaN goes, since R now drops NaN there.Tests
Six of the eleven
NNS.diff.customentries predate the R fix and are refilled; iterations move by one or two and derivatives in the 10th digit. The R helper gains asin_offsetfunction and an optional tol. The parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92. The exp derivative case now uses libm exp, as R does: with the new stop its inferred h is about 6e-9, and an ulp of difference in NumPy's SIMD exp moved it to 4.1e-9 on the CI runners.Mirrors OVVO-Financial/NNS#80, OVVO-Financial/NNS#81 and OVVO-Financial/NNS#83
Update
This push mirrors OVVO-Financial/NNS#103. For a two-valued numeric response in regression mode, a custom
obj_fnscores the final stack on the response values. It used to see the class codes 1 and 2 mixed with the values,{-5, 1, 2, 5}; it now sees{-5, 5}.Review
GPT-6.1 Sol reviewed this slice and found nothing. The 48 old/new stack cases were bit-for-bit identical on both paths.