Repository navigation
nns_arma_optim: errors, bias and intervals from the returned model (book ch25) - #174
Closed
gitRasheed wants to merge 1 commit into
Closed
gitRasheed wants to merge 1 commit into
gitRasheed wants to merge 1 commit into
Conversation
This was referenced Nov 2, 2026
gitRasheed
force-pushed
the
rasheed/book-arma-pi-lower
branch
from
November 2, 2026 04:12
423941e to
e44704c
Compare
gitRasheed
force-pushed
the
rasheed/book-optim-pi
branch
from
November 2, 2026 04:12
d9e100b to
65ce1ee
Compare
…im-pi-from-returned-model)
Mirrors R's optim-pi-from-returned-model branch (NNS.ARMA.optim).
errors and the bias were computed on the raw model with the selected
weights, before the shrink and nns.regress decisions, so once either was
taken the results came from a different model than errors and
bias.shift described. The interval was results +/- (|UPM.VaR(errors)| +
|bias|): symmetric, from the wrong model's errors, and widened by a bias
already subtracted.
The bias step now runs after shrink and nns.regress, on the validation
forecast of the returned model, and is kept only if it improves the
objective. errors is that forecast - actual before the bias, so
errors == results - bias.shift - actual on the validation window. The
interval adds lpm_var/upm_var((1 - pred_int)/2, 0, residuals) of the
validation residuals actual - (forecast - bias) to the results, as the
book builds intervals (ch. 17, 25, 26). The bias blocks in the weights
stage are removed; their result was always overwritten.
Before -> after:
AirPassengers[:132], training_set=105, seasonal_factor=[6, 12, 24], max
max|errors - (results - bias.shift - actual)| 107.26 -> 0
errors[:3] 19.40 71.65 114.35 -> 49.10 97.75 73.53
band results +/- 205.89 -> +65.76 / -130.63
two-cycle series (seed 33), h=10: bias.shift 0.7387 -> 0,
band results +/- 6.35 -> +4.12 / -5.85
Closes what the earlier optim and seasonality commits deferred: the four
optim selection cases compare errors, results, bias.shift and the lower
band again, and the nine NNS.VAR tests are no longer xfail. Four more
NNS.ARMA.optim.custom entries predate the R change and were refilled; the
other five came back identical.
gitRasheed
force-pushed
the
rasheed/book-arma-pi-lower
branch
from
November 2, 2026 23:09
e44704c to
f8bca9c
Compare
gitRasheed
force-pushed
the
rasheed/book-optim-pi
branch
from
November 2, 2026 23:09
65ce1ee to
a48cdc3
Compare
Contributor
|
Incorporated through NNS R 14.0 integration #192, merged as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
nns_arma_optimcomputederrorsand the bias on the raw model with the selected weights, before the shrink and nns.regress decisions, so once either was taken,resultscame from a different model thanerrorsandbias.shiftdescribed. The interval was results +/- (|UPM.VaR(errors)| + |bias|): symmetric, from the wrong model's errors, and widened by a bias already subtracted. The book bootstraps the forecast path and takes the lower bound from LPM.VaR (ch25:513-535, ch26:544, ch17:276). We mirror OVVO-Financial/NNS#88, which makes R follow the book. That PR changes the method, so the R author may close it; if so, this slice should be dropped too.Stacked on #173; merge with a merge commit.
Changes
errorsis that forecast minus actual, before the bias, so errors == results - bias.shift - actual on the validation window. On AirPassengers[:132] withtraining_set=105and lags [6, 12, 24], objective "max", the max of that difference goes from 107.26 to 0.lpm_var/upm_var((1 - pred_int)/2, 0, residuals)of the validation residuals to the results. On the same case the band goes from results +/- 205.89 to +65.76 / -130.63, and on the two-cycle seed-33 series from +/- 6.35 to +4.12 / -5.85, with bias.shift going from 0.7387 to 0.Tests
The four optim selection cases from #166 compare
errors,results,bias.shiftand the lower band again. Four moreNNS.ARMA.optim.customentries predate the R change and are refilled; the other five came back identical. The nineNNS.VARtests that were strict xfails since #170 pass and lose their marks. The parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92.Mirrors OVVO-Financial/NNS#88
Part of OVVO-Financial/NNS#71
Review
GPT-6.1 Sol reviewed this slice and raised one finding, which we rejected for now: validation residuals are taken before the zero clip, so the intervals describe a slightly different forecast when the bias-adjusted prediction goes negative. R (OVVO-Financial/NNS#88) does the same, and this slice matches it to 7 digits. The fix belongs in R first, then here.