Skip to content

ss: confidence intervals work when a group is constant - #187

Closed
gitRasheed wants to merge 1 commit into
rasheed/py-dydx-duplicate-bandsfrom
rasheed/py-ss-constant-group
Closed

gitRasheed wants to merge 1 commit into
rasheed/py-dydx-duplicate-bandsfrom
rasheed/py-ss-constant-group

Conversation

@gitRasheed

Copy link
Copy Markdown
Collaborator

R's NNS.SS with confidence.interval = TRUE bootstraps each group with NNS.meboot, which cannot rescale a constant series, so it failed whenever one group was constant. OVVO-Financial/NNS#148 (merged) uses the constant itself as that group's bootstrap. This slice mirrors it.

Stacked on #186; merge with a merge commit.

Changes

  • nns_ss (stochastic_superiority.py) tiles a constant group reps times instead of calling nns_meboot. Non-constant groups use the same derived seeds and nns_meboot calls as before, so their intervals don't change.
  • Before and after, with after equal to R main: nns_ss(rep(1, 30), rep(c(0, 1), 15), confidence_interval=TRUE) raised "function cannot be evaluated at initial parameters" and now gives p_star 0.75 inside [lower, upper]. Two constant groups give lower = upper = p_star (1 for 2 vs 1, 0.5 for 1 vs 1).

Tests

  • New invariants mirroring R's test-ss-constant-group.R: the one-constant-group interval in both orders, and the two-constant-group intervals. R's third test checks that non-constant groups still go through NNS.meboot with R's RNG stream; here the existing seeded parity tests cover that and still pass.
  • The invariant test that expected the error for two constant groups now checks the degenerate interval.
  • No cache entry moves.
  • Gates on the top of this stack (rasheed/ss-constant-group, which includes this slice), cache only: native 2935 passed, 4 skipped, 3 xfailed, 2 xpassed. With NNS_DISABLE_NATIVE=1: 2913 passed, 29 skipped, 2 xpassed. The 3 xfails are mode: break tied discrete modes by gravity, not their mean (mirrors R #111) #178's strict ones and the 2 xpasses the existing non-strict live-R VAR xfails. tests/parity/test_practical_examples.py against the usual R library: 9 passed, 2 xfailed, as on mode: break tied discrete modes by gravity, not their mean (mirrors R #111) #178. ruff check is clean on every changed file.

Review

GPT-6.1 Sol reviewed the #146, #147 and #148 mirrors together: SHIP.

Mirrors OVVO-Financial/NNS#148

…-group)

Mirrors R NNS.SS (OVVO-Financial/NNS ss-constant-group). With
confidence_interval=True each group is bootstrapped with NNS.meboot, which
cannot rescale a constant series, so nns_ss raised ("function cannot be
evaluated at initial parameters") whenever one group was constant. A
constant group's bootstrap is now the constant itself, repeated reps times.
Non-constant groups use the same seeds and NNS.meboot calls as before.

Before -> after (= R main):
  nns_ss(rep(1, 30), rep(c(0, 1), 15), confidence_interval=TRUE)
                         ValueError -> p_star 0.75 inside [lower, upper]
  nns_ss(rep(2, 10), rep(1, 15), confidence_interval=TRUE)
                         ValueError -> lower = upper = 1
  nns_ss(rep(1, 10), rep(1, 15), confidence_interval=TRUE)
                         ValueError -> lower = upper = 0.5

The invariant test that expected the error now checks the degenerate
interval. No cache entry moves.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head 0645d5d4189e67ec44d8571c5c9db20adca10384 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/py-ss-constant-group branch November 7, 2026 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant