Skip to content

nns_stack: score only the deployed fit for a single predictor (book ch24) - #176

Closed
gitRasheed wants to merge 1 commit into
rasheed/book-anova-grand-meanfrom
rasheed/book-stack-univariate-k
Closed

gitRasheed wants to merge 1 commit into
rasheed/book-anova-grand-meanfrom
rasheed/book-stack-univariate-k

Conversation

@gitRasheed

Copy link
Copy Markdown
Collaborator

With one predictor, the Method 1 fold loop ranked raw training observations by |x - x_test| and scored their k = 1..floor(sqrt(n)) means. The final fit is the univariate nns_reg, which ignores n_best, so the selected k was never deployed and OBJfn.reg scored a k-NN that is not returned. Book ch24 says the selected k is the one deployed (ch24:222). We mirror OVVO-Financial/NNS#90, which makes R follow the book. That PR changes the method, so the R author may close it; if so, this slice should be dropped too.

Stacked on #175; merge with a merge commit.

Changes

  • For one encoded predictor, ALL (the per-fold univariate nns_reg fit) is the only Method 1 candidate. OBJfn.reg is that fit's CV score and NNS.reg.n.best is NaN, R's NA. On pure noise with seed 5, OBJfn.reg goes from 154.14 to 177.29 and n.best from 6 to NaN; reg does not change.
  • For continuous responses reg, stack and the intervals do not change. With type="CLASS", the threshold is now tuned on the deployed fit's out-of-fold predictions, so classes can change: a balanced 3-class case goes from threshold 0.50 to 0.71 and reg from 1 1 2 3 to 1 1 2 2.

Tests

The univariate ALL entry and the labels n.best entry predate the R change and are refilled, and two entries are added for the new CLASS test. The parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92.

Mirrors OVVO-Financial/NNS#90

Review

GPT-6.1 Sol reviewed this slice and found nothing. Tied regression and classification cases matched R to 8.9e-16.

@gitRasheed
gitRasheed force-pushed the rasheed/book-anova-grand-mean branch from 81fd345 to 2035b9d Compare November 2, 2026 04:12
@gitRasheed
gitRasheed force-pushed the rasheed/book-stack-univariate-k branch from bfd84ef to 6b5132c Compare November 2, 2026 04:12
…ivariate-k)

Mirrors R's stack-univariate-k branch (NNS.stack). With one predictor
the Method 1 fold loop ranked raw training observations by |x - x_test|
and scored their k = 1..floor(sqrt(n)) means, but the final fit is the
univariate nns_reg, which ignores n_best: the selected k was never
deployed, OBJfn.reg scored a k-NN that is not returned, and
NNS.reg.n.best had no effect.

Now ALL, the per-fold univariate nns_reg fit, is the only candidate for
one encoded predictor (book ch. 24 tunes k for the multivariate step).
OBJfn.reg is that fit's CV score and NNS.reg.n.best is NaN (R's NA). For
continuous responses reg, stack and the intervals do not change; for
type="CLASS" the threshold is tuned on the deployed fit's out-of-fold
predictions, so returned classes can change.

Before -> after (= R):
  pure noise, one column, seed 5     OBJfn.reg 154.14 -> 177.29,
                                     n.best 6 -> NaN, reg unchanged
  3-class, one predictor, balance=True, test (2.1, 2.4, 4.9, 5.1)
                                     threshold 0.50 -> 0.71,
                                     n.best 1 -> NaN, reg 1 1 2 3 -> 1 1 2 2
  same, balance=False                threshold 0.58, n.best 3 -> NaN

Two entries (the univariate ALL case and the labels n.best case) predate
the R change and were refilled; two are added for the new CLASS test.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head ac6b6384b84576a065a4ac2cd0c3f1e4c80c4446 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/book-stack-univariate-k branch November 7, 2026 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant