Repository navigation
Mirror the R fixes to dy.dx, NNS.diff, dy.d_, NNS.stack and NNS.boost - #161
Closed
gitRasheed wants to merge 17 commits into
Closed
gitRasheed wants to merge 17 commits into
gitRasheed wants to merge 17 commits into
Conversation
This was referenced Nov 2, 2026
gitRasheed
force-pushed
the
rasheed/r-fixes-dominance-anova
branch
from
November 2, 2026 04:12
98b0faa to
22c4a06
Compare
gitRasheed
force-pushed
the
rasheed/r-fixes-regression-stack
branch
from
November 2, 2026 04:12
c5117b9 to
be4af9f
Compare
NNS partitions are discontinuous in X*: a last-ulp change can move a regression point. Three computations in the dimension-reduction path rounded differently from R, so folds partitioned differently, and the OOF objectives, n.best, weights and predictions drifted from R: - dim.red.method = "cor" used float64 two-pass Pearson on ranks. R's C_cor (cov.c) takes double deviations from long-double means (with a refinement pass), sums products sequentially in long double, and takes long-double square roots. On Boston, 6 of 13 coefficients were 1 ulp off. - The per-fold Method 1 design used `design @ coef` (OpenBLAS). R's `%*%` is reference-BLAS dgemv, which accumulates column by column. - The Method 2 fold candidates and the final projection used `@` too. R builds them with NNS_xstar_path_cpp, which accumulates in |coef|-descending order and divides by the running count of non-zero coefficients. The port now uses an R-exact Spearman cor, a column-sequential mat-vec, and a port of NNS_xstar_path_cpp. The path also builds all cumulative projections of a fold in one pass instead of one product per dimension count. Only the numerics change. Evidence: - cor: bit-identical to R on 180 of 180 coefficients (30 random designs with ties); the old code matched 74. - Audit A/B on 47 nns_stack cases vs NNS 13.3: no case moved away from R (changes elsewhere are at most 1e-13). OBJfn.reg goes from mismatched to 1e-15 on Boston default/pi/L1/folds3/seed7 and the CHAS factor cases. fac3_m12 stack goes from 4.4e-3 to 5e-15 and its weights from .01 to 3e-17 off. iris optimize.threshold=FALSE n.best goes from 3 (R 1) to 1, with weights matching. - New parity test (Boston, cv_size 0.25, folds 3 and 5): before, stack differs from R by up to 10.8; after, all outputs match at 1e-10. Remaining stack mismatches come from separate findings (NNS.dep ties, class coding, the "ALL" candidate) and are unchanged. nns_var still uses the float64 helper: its cor(matrix) call goes through a different R code path, and the coefficients are rounded to 2 digits.
With chas treated as a real two-level factor (two dummy columns, as in R) and nns_stack reproducing R's cor and X* rounding, the factor path matches the R reference again (stack RMSE, n.best, threshold and stack head).
… fix) The finite-difference step was gravity(|diff(x)|) * h with diff() taken on x in the order supplied. For unsorted x that is the typical gap between consecutive rows (about sd(x)), so the 2..sqrt(n) step multiples spanned the whole range and the derivative depended on row order. R now uses gravity(diff(sort(unique(x)))); do the same. Before -> after (y = x^3, 500 unsorted U(-2, 2) x (R seed 1), points -1.5 0 0.5 1.5, truth 6.75 0 0.75 6.75; = patched R): unsorted 3.731 3.881 3.748 3.726 -> 6.892 0.004 0.754 6.900 sorted 6.892 0.004 0.754 6.900 (unchanged) Sorted inputs without ties are unchanged. R's other dy.dx fixes need no Python change: one deleted dead code (no result change) and the other stops early without eval.point, which dy_dx already did. The dy.dx.numeric and dy.dx.overall shards are regenerated from the patched R.
…rror of the R fix) Each iteration solved for the inferred h with uniroot at its default tolerance (about 1.2e-4) and declared convergence when |h| < tol (1e-10), so uniroot's stopping point decided both the derivative accuracy and the "converged" flag: log'(5), tanh'(0.5) and (x^4 - 3x)'(-2) were reported converged with errors of 4e-7 to 6e-4. R now solves with uniroot(tol = tol / 10) and declares convergence when the bisection bracket on B is narrower than tol * max(1, |B|); nns_diff mirrors uniroot step for step, so do the same. Before -> after (DERIVATIVE error, iterations, converged; = patched R): sin'(1) -8.4e-08 29 1 -> 5.0e-10 30 1 (x^3)'(2) 8.6e-06 29 1 -> 4.4e-08 32 1 log'(5) -3.8e-07 28 1 -> 2.6e-10 30 1 tanh'(0.5) -1.0e-05 31 1 -> -8.8e-10 28 1 (x^4-3x)'(-2) 5.7e-04 30 1 -> 5.1e-08 32 1 Immediate exits (linear f, |x| at -1, point = 0) are unchanged. The NNS.diff.custom shard is regenerated from the patched R.
Follow-up to the mirror of the R dy.dx fix: docs/conventions.md still described the step as gravity(|diff(x)|) * h.
…e R fix) The nns_stack_numeric and nns_boost_numeric R helpers called set.seed(seed) and then NNS.stack / NNS.boost, which re-seed with their own default seed = 123L, so every R reference requested with seed 7, 9 or 42 actually ran with seed 123 (the seeded parity tests only compare structure, which hid it). Pass seed = as.integer(seed) to the R functions instead (123L when no seed is given, R's default). Cache keys do not include the script, so the NNS.stack.numeric and NNS.boost.numeric shards are regenerated from the patched R. Also pin the R fix (balancing samples within a one-member class instead of from 1:g): NNS Python's R sample() emulation already indexed the group, and with a single-member class the balanced Method 1 stack now matches the patched R exactly.
… (mirror of the R fix)
In the Method 1 n.best search the ALL candidate was scored out of fold
as a constant prediction, mean(training y), but when ALL won the final
fit was nns_reg(n_best="all") (multivariate) or the univariate nns_reg
fit, both non-constant estimators; model selection compared the local
candidates against a model that was never deployed, and OBJfn.reg
reported the SSE of a constant mean. Score ALL in every fold
with the estimator used for the final fit, as R now does.
Before -> after (pure-noise y, 160 training rows, 5 folds, seed 5;
= patched R):
one column: ALL wins, OBJfn.reg 126.865 (constant-mean SSE)
-> local k = 6 wins, OBJfn.reg 154.14
two columns: ALL wins, OBJfn.reg 126.865 -> ALL wins, OBJfn.reg 127.398
(out-of-fold SSE of nns_reg(n_best="all"))
The selected n.best and OBJfn.reg change wherever ALL was involved in
the decision; this also fixes the nns_var(objective="max") multivariate
stage, which now matches the patched R. The NNS.stack.*, NNS.boost.* and
NNS.VAR* shards are regenerated from the patched R (no further entry
changes).
With the stack "all" fix (the univariate "all" candidate is scored with the model it deploys) a k candidate wins on the n.best test data, so the test now compares n.best and the predictions with R instead of asserting NA. Re-record that entry from R 13.3 + the patch series.
…or of the R fix)
nns_boost treats a numeric response as a regression unless
type="CLASS", but its final nns_stack call auto-classifies a numeric
response with exactly two unique values and returns class codes 1/2,
which were passed through as regression estimates, so a {0, 1} or
{-5, 5} response came back as 1s and 2s. When the final stack
reports class levels for a regression boost, map its codes (results
and pred.int) back to the response values, as R now does.
Before -> after (40 points, y = -5/5 by x > 0.3, 5 test rows,
learner_trials=5, epochs=3, seed 1; = patched R):
results 1 1 1 2 2 -> -5 -5 -5 5 5
(y = 0/1 likewise 1/2 -> 0/1)
Continuous and factor responses are unchanged. The NNS.boost.* shards
are regenerated from the patched R (no existing entry changes).
…he R fix) R's native multivariate setup reduced noise.reduction = "mode" cells with the exact most-frequent value (smallest on ties), while its pure-R reference (and the order = "max" path) uses NNS.mode(discrete = FALSE). NNS Python follows the reference, so it disagreed with R's default path; R now calls NNS.mode in the native reducer too. No Python change. New parity case (300 rounded 2-D points): nns_reg(x, y, noise_reduction="mode") R2 0.81303 R 13.3 default 0.78043 -> patched R 0.81303 (= Python, = pure-R path)
… R fix) dy_d estimated f(x +/- h e_wrt) with nns_stack on the equal-weight synthetic regressor X* = rowMeans(x), for both the training design and the evaluation points, so every estimate depended only on mean(x) and each regressor received about the index slope / p whatever its true effect: for y = 2*x1 the derivative wrt the irrelevant x2 was about 1. R now uses the multivariate NNS.reg fit on the actual regressors (NNS.reg(x, y, point.est = pts, order = NULL)); do the same. Before -> after (y = 2*x1, 500 R-seed-1 normal pairs, points (-1,0), (0,1), (1,-1); = patched R): dy/dx1 (truth 2): 0.694 0.524 0.666 -> 1.184 1.407 1.079 dy/dx2 (truth 0): 0.644 0.753 0.723 -> 0.065 0.037 -0.067 Signs and relative effects are now right; magnitudes stay attenuated by the wide v0.5.7 bandwidths, left for a follow-up R change. Runtime is unchanged or lower (100 x 2, "mean" 0.23 -> 0.20 s, "obs" 0.14 -> 0.09 s). dy_d_best, a Python-only function that always returned exactly what dy_d returns, becomes a deprecated alias of dy_d that emits a DeprecationWarning. dy_d keeps messages=True, R's default. The dy.d.scalar shard is regenerated from the patched R and new parity cases pin the irrelevant-regressor behaviour.
…key is portable x**3 is not correctly rounded on every numpy build, so the y values, and with them the hashed R-cache key, differed in the last bits between machines and the row-order entry missed. x is now drawn from integers / 1024 in [-2, 2], where x * x * x is exact, as the bandwidth test already does. Unpatched NNS 13.3 gives 3.696 3.877 3.790 3.707 on these inputs, so the comment's range changes. The new entry comes from the patched R; the old entry is removed.
With the stack fixes mirrored, the case matches R again.
The new B-bracket stopping rule moves the quintic case from Inferred h 8.5e-11 after 32 iterations to 1.187e-09 after 31, in R and Python alike.
They point at notes that don't live in this repo.
Data such as {0, 2.2e-308, 1} make R's NNS.stack fail with "No candidate
produced a finite out-of-fold objective" or a nonfinite point.est, and the
port now raises the same errors, so the finite-output property can't hold
there.
gitRasheed
force-pushed
the
rasheed/r-fixes-dominance-anova
branch
from
November 2, 2026 23:09
22c4a06 to
ed0c496
Compare
gitRasheed
force-pushed
the
rasheed/r-fixes-regression-stack
branch
from
November 2, 2026 23:09
be4af9f to
898260b
Compare
Contributor
|
Incorporated through NNS R 14.0 integration #192, merged as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NNS-python copied five regression-side R bugs.
dy.dxdepended on row order,NNS.diffreported convergence with errors up to 5.7e-4,dy.d_differentiated a row-mean index instead of the fit, the stack's ALL candidate was scored as a constant mean but deployed as a different model, andNNS.boostreturned class codes for a two-valued numeric response. We mirror each, and makenns_stackreproduce R'scorand X* rounding so partitions match R.Stacked on #160; merge with a merge commit.
Changes
nns_stack's dim.redcorfollows R'sC_corlong-double arithmetic, the Method 1 fold design uses a column-sequential mat-vec like R's reference dgemv, and Method 2 uses a port ofNNS_xstar_path_cpp.coris bit-identical to R on 180 of 180 coefficients (74 before), and on Boston with folds 3 and 5 the stack differed from R by up to 10.8 and now matches at 1e-10.dy_dxtakes the step from the spacing of sorted unique x. For y = x^3 on 500 unsorted points, the derivatives at (-1.5, 0, 0.5, 1.5) go from 3.731 3.881 3.748 3.726 to 6.892 0.004 0.754 6.900, the sorted-input values.nns_diffsolves for h at tol / 10 and stops when the bracket on B is narrower than tol * max(1, |B|). The (x^4 - 3x)'(-2) error goes from 5.7e-4 to 5.1e-8.dy_ddifferentiates the multivariatenns_regfit on the actual regressors. For y = 2 x1 at three points, dy/dx2 (truth 0) goes from 0.644-0.753 to between -0.067 and 0.065, and dy/dx1 (truth 2) from 0.524-0.694 to 1.079-1.407. Magnitudes stay attenuated by the wide bandwidths.dy_d_best, which always returned whatdy_dreturns, is a deprecated alias.nns_var(objective="max")multivariate stage.nns_boostmaps the final stack's class codes back to the response values: a -5/5 response came back as 1 1 1 2 2 and now as -5 -5 -5 5 5.NNS.stack/NNS.boost, which re-seeded with 123 and so ignored seeds 7, 9 and 42. New cases pin R's single-member-class balancing fix and multivariatenoise.reduction="mode", which needed no Python change.Tests
New parity cases for each fix against R 13.3 with the audit's R patches; the
NNS.stack.numeric,NNS.boost.numeric,NNS.diff.custom,dy.dx.numericanddy.d.scalarshards are regenerated from it. Thedy_dxrow-order inputs move to a 1/1024 grid so the cache key is portable. The univariate n.best labels test compares n.best and predictions with R instead of asserting NA. The stack property strategy now skips nonzero values below 1e-100: on data such as {0, 2.2e-308, 1}, R'sNNS.stackraises "No candidate produced a finite out-of-fold objective" or a nonfinite point.est, and the port now raises the same errors.Two xfail marks are dropped. The Boston factor-path practical example, xfailed since #155, passes with the
corand X* rounding fix, andtest_public_nns_var_objective_max_matches_rpasses with the ALL fix.Mirrors OVVO-Financial/NNS#80, OVVO-Financial/NNS#83 and OVVO-Financial/NNS#84
Part of OVVO-Financial/NNS#65
Review
GPT-6.1 Sol reviewed this slice and raised one finding, which we rejected: the intercept-scaled stop in
nns_diffclaims convergence with a 2% error once a constant is added to f. The R branch this slice follows has the same rule, and #168 mirrors R's fix. At the top of the stack the reviewer's case returns 0.001999999995 and does not claim convergence.