Skip to content

Fix the Boston stack parity tests, the curve-fitting gist and CI tool pins - #148

Merged
OVVO-Financial merged 7 commits into
mainfrom
rasheed/test-ci-prep
Nov 7, 2026
Merged

OVVO-Financial merged 7 commits into
mainfrom
rasheed/test-ci-prep

Conversation

@gitRasheed

@gitRasheed gitRasheed commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

The two Boston nns_stack parity tests fail on main because they compare different models. The Python side pinned cv_size=0.25, while the R reference calls NNS.stack with its default 5-fold CV.size. We fix that and a few test and CI problems that the slices above need out of the way.

This builds on main and is the first of 29 stacked PRs; merge each one with a merge commit.

Changes

  • The Boston tests call nns_stack with R's default CV.size. Stack RMSE was 3.7147 against R's 3.7395, and Python and R now agree to about 6e-14 on reg, dim.red, stack, n.best and threshold. The test also stops overwriting the dimension-reduction threshold with R's value, which hid the assertion.
  • gists/curve_fitting called nns.reg, which does not exist, inside a try/except that silently skipped the NNS panel. It now calls nns_reg(plot=False) and runs end to end.
  • CI installs ruff 0.16.3 and mypy 2.3.1, the dev-group pins that pyproject.toml says CI gates on, instead of the latest release. The docs group is capped at mkdocs<2, which mkdocs-material says it does not support.

Tests

The regression and causation property strategies no longer draw subnormal floats, where an interpolation slope overflows to inf * 0 = NaN. The causation property also stays above 1e-100, because at 1e-249 NNS.norm's division by the column means overflows and R and Python both stop with "variable must be finite". No library code changes.

The parity cache encodes unpatched R 13.3. test_nns_stack_ts_test_method1_shape can be OOM-killed at about 8 GB on a small machine, here as on main; GitHub's runner survives it, and it goes away at #157.

Review

GPT-6.1 Sol reviewed this slice and found nothing. It ran the regression and causation properties and the Boston comparisons against live R 13.3 on both paths.

The R reference scripts call NNS.stack with the default CV.size (5-fold),
but the Python side pinned cv_size=0.25, so the two tests compared different
models (stack RMSE 3.7147 vs 3.7395) and failed. With matching settings
Python and R agree to ~6e-14 on reg, dim.red, stack, n.best and threshold.

Also stop overwriting the dimension-reduction threshold with R's value: the
Python threshold already equals R's, and the override masked the assertion.
The gist called nns.reg, which does not exist, and wrapped the result in a
try/except that silently skipped the NNS panel. Call nns_reg(plot=False) and
plot Fitted.xy directly; the script now runs end to end.
pyproject.toml says the dev pins (ruff 0.16.3, mypy 2.3.1) are the versions
CI gates on, but the workflows installed whatever was latest, so a new
upstream release could break CI or disagree with local runs. Also cap the docs
group at mkdocs<2: mkdocs-material warns MkDocs 2.0 is incompatible.
test_nns_reg_classification_bounds_hold was flaky: hypothesis occasionally
drew subnormal x (e.g. 2.2e-309 next to zeros), where an interpolation slope
overflows to inf * 0 = NaN. That edge is tracked as its own issue; the shape
and bounds properties are about normal-range data.
The causation property strategy hit the same subnormal edge as the
regression one (2.2e-313 next to zeros); exclude subnormals there too.
At ranges like 1e-249 NNS.norm's division by the column means overflows and
both R and Python stop with "variable must be finite" (NNS.caus is not
scale invariant). Restrict the property to ranges above 1e-100.
They point at notes that don't live in this repo.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head 45743036b9a084d14badf127d91fb1f5404650e6 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions
github-actions Bot deleted the rasheed/test-ci-prep branch November 7, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants