Skip to content

Mirror the R fixes to NNS.ARMA, NNS.ARMA.optim and NNS.VAR - #159

Closed
gitRasheed wants to merge 14 commits into
rasheed/r-fixes-moments-dependencefrom
rasheed/r-fixes-timeseries
Closed

gitRasheed wants to merge 14 commits into
rasheed/r-fixes-moments-dependencefrom
rasheed/r-fixes-timeseries

Conversation

@gitRasheed

@gitRasheed gitRasheed commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

NNS-python copied the R 13.3 forecasting bugs for parity: the NNS.VAR multivariate stage was an in-sample refit of the last h observed rows, method="means" returned zeros, nonlinear forecasts were clipped at 0, weights were applied twice, and NNS.ARMA.optim scored models other than the one it returned. We mirror the patched R for each and regenerate the affected cache shards from it.

Stacked on #158; merge with a merge commit.

Changes

  • nns_var evaluates the multivariate stack on the lag-matrix rows of the forecast period, built from the series extended with the univariate forecasts. Before, adding 1000 to every univariate forecast left multivariate bit-identical. Every nns_var call changes multivariate and ensemble; univariate is unchanged.
  • method="means" returns the seasonal means. AirPassengers, h = 4, lag 12 goes from 0 0 0 0 to 241.75 235 270.17 267.08.
  • Nonlinear estimates honour negative_values. On a signed sine, months 7-11 go from 0 to -1.60 -2.96 -3.29 -2.55 -1.15.
  • Weights are applied once and rescaled to sum to 1, one weight per lag is required, and weights="equal" works with numeric lags. lin with weights [1, 1] on AirPassengers gave 1660.6 1557.6 1812.3, and now 415.1 389.4 453.1.
  • Zero or non-finite seasonal weights fall back to equal weights. nns_arma([5] * 60, h=3) goes from 0 0 0 to 5 5 5, and the AirPassengers dynamic case from 441.71 0 0 0 to 441.58 406.03 455.48 476.33.
  • nns_arma_optim scores "both" as nns_arma(method="both"), evaluates the equal-weights candidate, and applies the nns.regress decision to results.
  • Fractional-degree lpm_var/upm_var solve with uniroot instead of copying optimize() at its default tolerance. On NVDA returns at degree 1.5 the max inverse residual goes from 4.2e-4 to 3.3e-8.
  • h must be one finite whole number >= 1, with R's message. h = 2.5 was truncated to 2.
  • seasonal_factor=True honours best_periods, as R does. Python used to ignore it on purpose because R's path could forecast 0; with the equal-weights fallback that no longer happens. best_periods=1 gives the same result as before.
  • The linear recursion step reuses one index array and calls the native fast_lm directly, with bit-identical estimates. A 5-series daily nns_var at n = 500 goes from 5.2-6.0 s to 3.8-4.9 s.

Tests

New parity cases for each fix, against R 13.3 with the audit's R patches; the NNS.ARMA, NNS.ARMA.optim.custom, NNS.VAR, NNS.VAR.custom, LPM.VaR and UPM.VaR shards are regenerated from it. A new nns_var invariant moves the univariate forecasts and requires multivariate to move.

The eight NNS.VAR xfails from #158 pass again and lose their mark. test_public_nns_var_objective_max_matches_r gets the mark instead, because objective="max" reaches the stack's ALL candidate; #161 drops it.

Mirrors OVVO-Financial/NNS#77, OVVO-Financial/NNS#78, OVVO-Financial/NNS#79 and OVVO-Financial/NNS#85
Part of OVVO-Financial/NNS#60, OVVO-Financial/NNS#63 and OVVO-Financial/NNS#64

Review

GPT-6.1 Sol reviewed this slice and raised three findings. We took two and rejected one.

  • Taken: docs/conventions.md still promised zero or NaN constant forecasts and rejected "equal" weights. It now describes the equal-weight fallback and character weights.
  • Taken: the "both-scored" ARMA fixture could not tell the new scoring from the old one. We added full AirPassengers with h = 12 and seasonal factors 1, 2, 12, where R picks nonlin (obj.fn 0.11104) and the old averaged scoring picks "both".
  • Rejected: when equal weights win, the winning score isn't updated, so a worse shrink forecast can replace it. The R branch this slice follows has the same behaviour; the R fix came later and ARMA optim scoring and dynamic weights: mirror the later R fixes #166 mirrors it. At the top of the stack obj.fn equals the equal-weights MSE (6.7635).

The fix commits have not been re-reviewed.

In the recursive path the means estimate was computed and then thrown
away, so nns_arma(method="means") always returned zeros, as in R 13.3.
R now returns it; do the same.

Before -> after (= patched R):
  nns_arma(AirPassengers, h=4, seasonal_factor=12, method="means")
      0 0 0 0 -> 241.75 235 270.17 267.08 (241.75 = mean of the Januaries)
  same, seasonal_factor=[12, 24]  0 0 0 0 -> 236.15 230.13 266.97 262.74
  nns_arma(1:20, h=5, seasonal_factor=4, method="means")
      0 x 5 -> 9 10 11 12 9

Other methods are unchanged. The NNS.ARMA shard is regenerated from the
patched R (the two constant-series entries keep their 13.3 values until
the constant-series fix is mirrored).
…e R fix)

The per-lag nonlinear estimates were always clipped with max(0, .),
while the linear branch is guarded by negative_values, so every nonlin
(and the nonlin half of "both") forecast of returns, spreads or demeaned
data was floored at 0, even with negative_values=True or when negative
values were auto-detected. R now guards it; do the same.

Before -> after (3 sin(2 pi t/12) + 0.3 cos(t), t = 1..120, h = 12,
seasonal_factor=12, method="nonlin"; = patched R):
  months 7..11  0 0 0 0 0 -> -1.60 -2.96 -3.29 -2.55 -1.15

Series without negative values are unchanged. The NNS.ARMA shard is
regenerated from the patched R (constant-series entries as before).
In the recursive path the lin and nonlin estimates are already
sum(estimate * weights), and "lin"/"nonlin" multiplied by the weights a
second time, so forecasts were scaled by sum(w): weights [1, 1] gave 4x
for lin/nonlin and 2x for "both" (which weights once), and a single lag
with weights 0.5 gave 0.25x for nonlin but 0.5x for the vectorised lin
path; weights="equal" (documented) raised TypeError with a numeric
seasonal_factor. R now applies the weights once, treats user
weights as relative (rescaled to sum to 1), requires one weight per
lag, and accepts "equal" for numeric seasonal factors; do the same
(finite weights that do not sum to 0, else ValueError).

Before -> after (AirPassengers, h=3, seasonal_factor=[12, 24]; = patched R):
  lin    weights [1, 1]   1660.6 1557.6 1812.3 -> 415.1 389.4 453.1
  nonlin weights [1, 1]   1727.9 1623.9 1821.5 -> 432.0 406.0 455.5
  both   weights [1, 1]    847.1  795.4  908.5 -> 423.6 397.7 454.3
  sf=12, weights [0.5], nonlin  112.6 105.4 111.5 -> 450.0 421.3 445.9
  weights="equal"               TypeError -> 432.0 406.0 455.5
Weights that sum to 1 (and weights=None) give unchanged results. The
NNS.ARMA shard is regenerated from the patched R.
…R fixes)

ARMA.seas.weighting(FALSE, .) weights the selected periods by
(VarCV - CV) / sqrt(lag), normalised by their sum. When every selected
row has CV == VarCV (the no-seasonality sentinel, modulo-added periods,
a constant series) the sum is 0 and all weights were 0, so the forecast
was exactly 0; with a numeric seasonal_factor a constant
series gave 0/0 weights and NaN forecasts. R now falls back to equal
weights in both cases.

Python had the same zero weights in _arma_seas_weighting, which its own
dynamic path reaches: nns_arma(AirPassengers, h=4, dynamic=True,
seasonal_factor=False, best_periods=2, modulo=12) gave 441.71 0 0 0.
Use equal weights there and for non-finite numeric-lag weights, and drop
the special case that returned zeros for a constant series.

Before -> after (= patched R):
  white noise (mean 10), nns_arma(x, h=4, modulo=7)  (13.3 R: 0 0 0 0)
                                         Python 7.94 10.68 4.88 12.54 (unchanged)
  AirPassengers, dynamic, sf=False, best_periods=2, modulo=12
                                         441.71 0 0 0 -> 441.58 406.03 455.48 476.33
  nns_arma([5] * 60, h=3)                0 0 0 -> 5 5 5
  nns_arma([5] * 60, h=3, seasonal_factor=12 or [12, 24], any method)
                                         NaN NaN NaN -> 5 5 5
  nns_arma([-2] * 30, h=2, seasonal_factor=7)  NaN NaN -> -2 -2

The NNS.ARMA shard is regenerated from the patched R; the new cases also
pin the R fixes for the best period with seasonal_factor=True,
best_periods=None and for best_periods/modulo with dynamic=True,
which Python already implemented.
…gress (mirror of the R fixes)

Three NNS.ARMA.optim selection bugs that nns_arma_optim copied:
- "both" stage: with the linear approximation it scored the
  average of the separately forecast lin and nonlin paths, but the
  returned model is nns_arma(method="both"), which averages inside the
  recursion. Score nns_arma(method="both") directly.
- equal weights: for multi-period selections the equal-weights
  score was computed from the weights=None forecast, so it always equalled
  the search score and $weights was always None. Forecast with equal
  weights, score that (NaN counts as worst) and keep it when strictly
  better; the bias test then compares with that score as R does.
- nns.regress: when the smoothed series won, the decision was
  reported but $results still came from the raw series. Use the smoothed
  series with shrink=True for the results and report shrink=True.

Before -> after (= patched R):
  AirPassengers, h=12, sf=[1, 2, 12]
      both [12] obj.fn 0.10516, 453.3 425.0 465.1 495.9
      -> nonlin [12] obj.fn 0.11104, 452.4 423.7 448.3 502.0
  two-cycle series (R seed 9 noise), h=10, sf=[4, 6, 7, 12, 14, 24]
      lin [12, 7] weights None, 51.68 51.23 50.36
      -> weights [0.5, 0.5], 51.56 51.39 50.70
  AirPassengers[1:132], training_set=120, sf=[6, 12, 24], objective="max"
      lin [6], nns.regress True, shrink False, 349.2 343.2 332.2 305.8 (raw series)
      -> shrink True, 318.4 321.2 323.8 326.5 (smoothed-series forecast)
  AirPassengers[1:132], training_set=120, sf=[6, 12, 18, 24]: both [24]
      0.02359 (unchanged)

R's other optim fixes need no Python change: negative.values in
the lin search and the exhaustive search without the linear
approximation were already how Python worked; the exhaustive case is pinned.
The NNS.ARMA.optim.custom helper gains objective / linear_approximation
(sent to R only when not default, so existing keys are kept) and the
shard is regenerated from the patched R (no existing entry changes).
…the R fix)

The multivariate nns_stack was trained on the observed lag-matrix rows
and evaluated on the last h of those same rows (R's tail(IV, h)), so
the "multivariate forecast" was an in-sample refit of the last h
observed periods and carried no information about the forecast period;
the ensemble was half a replay of the observed tail. Python
copied this on purpose (docs/conventions.md). R now evaluates on the
last h rows of the lag matrix built from the series extended with the
univariate forecasts (the rows for n+1..n+h); do the same.

Evidence: with the old evaluation, adding 1000 to every univariate
forecast left `multivariate` bit-identical; now it changes (new
invariant test). The univariate stage is unchanged; the multivariate
and ensemble outputs of every nns_var call change.

The NNS.VAR and NNS.VAR.custom shards are regenerated from the patched
R. test_public_nns_var_objective_max_matches_r still differs in the
multivariate stage (objective="max" reaches the NNS.stack Method 1
"ALL" candidate, an R fix mirrored later); its entry already holds
the patched-R value.
For degrees without the exact prefix-polynomial inversion (fractional
degrees) lpm_var/upm_var copied R 13.3's optimize() on |ratio(t) - p|
with its default absolute tolerance (eps^0.25, about 1.2e-4 in x units),
so the inverse was off by up to 0.8% relative at 5% on daily returns and
p = 0 / 1 did not return the sample min / max. The ratio is monotone in
t, so, as the patched R does, solve ratio(t) - p = 0 with uniroot
(R's zeroin2, already ported in nns.diff) on [min(x), max(x)] at
tol = sqrt(eps) * diff(range(x)), returning an end of the support when
the root is at or beyond it.

Before -> after (NVDA daily returns, n = 4231, 12 levels incl. 0 and 1):
  degree 1.5: max |lpm_ratio(lpm_var(p)) - p| 4.2e-4 -> 3.3e-8
  degree 0.5: 5.7e-5 -> 2.1e-8
  lpm_var([0, 1], 1.5, x): -0.187481 0.297990 -> -0.187559 0.298067
                           (= min, max)
  vs patched R, degrees 0.5 / 1.5 / 2.5, NVDA and t(4) samples:
  identical (max abs diff 0.0); before up to 7.7e-5.
  8 levels at degree 1.5: 11.0 ms -> 8.0 ms.
Integer degrees use the exact path and are unchanged. The LPM.VaR /
UPM.VaR cache entries the tests read were regenerated from the patched
R: only the degree-1.5 entries change; degree-0.5 entries are added.
nns_arma converted h with int(h), so h = 2.5 was silently truncated to
2, h = True gave one forecast, h = "3" was accepted, and NaN / inf /
lists failed with Python conversion errors; h = 0 and h = -1 raised
"h must be a positive integer.". R 13.3 ran for (j in 1:h) (h = 0 gave
one forecast, h = 2.5 two) and now stops with "[h] must be a positive
integer." unless h is one finite whole number >= 1. Validate h the same
way (whole floats and NumPy integer scalars are accepted).

Before -> after (AirPassengers, seasonal_factor = 12; R = patched R):
  h = 2.5    450.045 421.326        -> "[h] must be a positive integer."
  h = True   450.045                -> same error (R: same)
  h = "3"    450.045 421.326 445.926 -> same error (R: same)
  h = nan    "cannot convert float NaN to integer" -> same error
  h = 0, -1  "h must be a positive integer." -> R's message
  h = 3, 3.0 450.045 421.326 445.926 (unchanged, = R)

The NNS.ARMA* cache entries the tests read were regenerated from the
patched R. One NNS.ARMA.optim.custom entry changes (the nns.regress
case, by up to 2e-3): its NNS.reg(smooth = TRUE) step sets the spline
spar from NNS.dep / NNS.copula, which the patched R changed (dependence
of (1:132, AirPassengers[1:132]) 0.80012 -> 0.79912). nns_reg
smooth = TRUE now differs from that R by 0.032 (smooth = FALSE: 5e-13),
so test_nns_arma_optim_selection_matches_patched_r[regress-max-...]
waits for the degree-0 co-moment / NNS.copula mirrors (native,
done separately); with them the refilled entry passes.
R's NNS.ARMA turns seasonal.factor = TRUE into FALSE whenever
best.periods is non-NULL (the default is 1), so TRUE with
best.periods = k weights the k best NNS.seas periods; only
best.periods = NULL keeps the TRUE path (the best period, after the
R fix). nns_arma always took the TRUE path and silently ignored
best_periods. The difference used to be deliberate because R's
FALSE path forecast 0 when all selected weights were 0/0; the patched
R (and nns_arma since the mirror of that R fix) falls back to
equal weights there, so the R semantics are now safe and honour an
explicitly passed argument. With best_periods = 1 the FALSE path on a
single period gives weight 1, the same as before.

Before -> after (AirPassengers, h = 4; R = patched R):
  seasonal_factor=True, best_periods=2, nonlin
    333 421 458 408 -> 388.349 432.620 428.644 440.949 (= R,
    = seasonal_factor=False, best_periods=2)
  best_periods=3        333 421 458 408 -> 387.891 425.761 414.048 457.768 (= R)
  best_periods=2, lin   333 421 458 408 -> 367.249 426.198 428.644 425.583 (= R)
  best_periods=2, both  333 421 458 408 -> 377.799 429.409 428.644 433.266 (= R)
  best_periods=1 / None 333 421 458 408 (unchanged, = R)
New parity cases (incl. modulo and dynamic = TRUE) add NNS.ARMA cache
entries from the patched R; existing entries are unchanged.
Each recursive step of nns_arma built every lag's component series and
index vector through _generate_vectors (two allocations and a copy per
lag), then fitted each through _fast_lm, which re-validates its inputs and
looks the native kernel up again. In NNS.VAR on daily returns (129-198
seasonal candidates per column, searched greedily by
NNS.ARMA.optim) that was 580k _fast_lm calls and 156k _generate_vectors
calls for a 5-series n=500 run.

The linear fits now go through _linear_forecasts: the same strided
component, an index taken as a prefix view of one 1..N array, and the same
native fast_lm call, so every estimate is bit-identical.
_generate_vectors only runs when the nonlinear, means or shrink branches
need the components. Without the native kernel, or for an empty
component (which raises), the old per-lag path is used.

Bit-identical: an A/B over 432 nns_arma configurations (lin/both/means/
nonlin, seasonal True or 1-3 lags, dynamic, shrink, pred_int; AirPassengers,
VIX, simulated returns, a short series that raises) and nns_arma_optim on
VIX gave identical outputs and errors; the NNS.VAR ensemble below is
identical; new invariant test.

Timings (best of 3-5, shared 4-core box; noisy), before -> after:
  nns_var daily 5 x 500, h=5, tau=2       5.2-6.0 s -> 3.8-4.9 s
  nns_arma lin, VIX n=1000, h=200, 24 lags   39 ms -> 23 ms
  nns_arma_optim VIX n=1000, sf=1:24     0.20-0.23 s -> 0.18-0.22 s
The rest of NNS.VAR is the nonlinear branch (nns_reg per lag and step,
~50%), and optim on VIX is dominated by the native copula behind
nns_reg's default order; neither changes here. Capping VAR's seasonal
candidates (the other proposed speed-up) would change results, so it is
left for a coordinated R change. Caching OLS sums across steps is not
possible bit-identically: the component used at each step has gained a
point since its previous use, and the native fit accumulates in long
double.
The eight marked cases match R again with the forecast-row fix. The
objective="max" case now needs the stack fixes as well, so it is the one
left marked xfail.
conventions.md still promised zero/NaN constant forecasts and rejected
weights="equal"; nns_arma(rep(5, 60), h = 3) gives 5 5 5 and "equal" works.
With full AirPassengers, h = 12, seasonal_factor [1, 2, 12], the old averaged
lin/nonlin score picks "both"; the returned-model score picks nonlin (0.11104) as R.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head ea29dbf08877f6d7c3ebfdc622e8c7a32317f404 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/r-fixes-timeseries branch November 7, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant