Repository navigation
ARMA optim scoring and dynamic weights: mirror the later R fixes - #166
Closed
gitRasheed wants to merge 3 commits into
Closed
gitRasheed wants to merge 3 commits into
gitRasheed wants to merge 3 commits into
Conversation
This was referenced Nov 2, 2026
gitRasheed
force-pushed
the
rasheed/mirror-dependence
branch
from
November 2, 2026 04:12
3bace5c to
2cd45bb
Compare
gitRasheed
force-pushed
the
rasheed/mirror-timeseries
branch
from
November 2, 2026 04:12
99810a0 to
d581bd9
Compare
…hrink (mirror of the R fixes)
Three fixes from R NNS.ARMA.optim (arma-optim-scoring):
- When equal weights win, obj.fn and the later shrink and nns.regress
comparisons used the weights = None score. Take the equal-weights score.
- The nns.regress candidate smoothed the whole series and was scored on
the validation window, so the smooth had seen the values it was scored
against. Smooth only the training window for the decision and the
validation-window results; the out-of-sample forecast (h given) still
smooths the full series.
- The smoothed candidate was compared with the score before shrink.
Compare it with the best score so far.
Before -> after (decisions and obj.fn = R):
equal-weights case (two-cycle series, seed 33), obj.fn
1.054731 -> 1.021186
random walk seed 13, training_set 80, nns.regress True -> False
random walk seed 16, nns.regress stays True, results
39.095 39.301 39.504 -> 39.173 39.348 39.523
The two changed NNS.ARMA.optim.custom entries are refilled from R and two
are added. R's refilled values also carry optim-pi-from-returned-model
(bias and errors from the returned model), so for these cases the test
compares the search outcome (method, periods, weights, obj.fn, shrink,
nns.regress) and leaves the value keys to that change.
…tep (mirror of the R fix)
The dynamic loop re-resolved the lags at each step with weights=None, so
user weights were silently replaced by the NNS.seas weights. As in R
NNS.ARMA (arma-forecasts), numeric weights with dynamic=True now raise
(they belong to fixed lags, like a numeric seasonal_factor, which already
raises), and weights="equal" is applied to whichever lags each step
selects.
Before -> after, 100 + t + 15 sin(pi t / 6), t = 0..59, h = 3,
seasonal_factor=False, best_periods=2, dynamic=True:
weights="equal" 181.788 184.092 186.380
-> 181.495 183.784 186.057 (= repeated h = 1 calls)
weights=[1, 0] weights ignored -> ValueError (R: error)
weights=None and static calls are unchanged.
walk-16 compared only the search outcome, so a smooth that saw the validation window still passed; results - bias.shift now matches R to 1.8e-11, the mutant is 0.14 off.
gitRasheed
force-pushed
the
rasheed/mirror-dependence
branch
from
November 2, 2026 23:09
2cd45bb to
df1f9c5
Compare
gitRasheed
force-pushed
the
rasheed/mirror-timeseries
branch
from
November 2, 2026 23:09
d581bd9 to
d2767c8
Compare
Contributor
|
Incorporated through NNS R 14.0 integration #192, merged as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four fixes from the R
NNS.ARMA.optimandNNS.ARMAPRs that came after the first mirror. When equal weights won,obj.fnand the shrink and nns.regress comparisons still used the weights = None score. The nns.regress candidate smoothed the whole series and was scored on the validation window, so the smooth had seen the values it was scored against. Withdynamic=True, user weights were silently replaced by the NNS.seas weights.Stacked on #165; merge with a merge commit.
Changes
obj.fnis the equal-weights score when equal weights win. On the two-cycle seed-33 series it goes from 1.054731 to 1.021186.training_set=80, nns.regress goes from True to False.dynamic=Trueraise, as in R.weights="equal"applies to whichever lags each step selects: the forecast goes from 181.788 184.092 186.380 to 181.495 183.784 186.057, which equals repeated h = 1 calls.weights=Noneand static calls are unchanged.Tests
The two changed
NNS.ARMA.optim.customentries are refilled from R and two are added. R's refilled values also carry the book-alignment change in OVVO-Financial/NNS#88, so for now these cases compare the search outcome (method, periods, weights, obj.fn, shrink, nns.regress) and leave the value keys to #174. A new invariant covers dynamic weights. The parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92.Mirrors OVVO-Financial/NNS#77 and OVVO-Financial/NNS#78
Part of OVVO-Financial/NNS#63 and OVVO-Financial/NNS#64
Review
GPT-6.1 Sol reviewed this slice and raised one finding, which we took: the new walk-16 case had no forecast assertion, so reintroducing the validation-window leak still passed. A separate test now asserts
results - bias.shift, which matches R to 1.8e-11; the leaking variant is 0.14 off and fails. The fix commit has not been re-reviewed.