Skip to content

nns_arma_optim: errors, bias and intervals from the returned model (book ch25) - #174

Closed
gitRasheed wants to merge 1 commit into
rasheed/book-arma-pi-lowerfrom
rasheed/book-optim-pi
Closed

gitRasheed wants to merge 1 commit into
rasheed/book-arma-pi-lowerfrom
rasheed/book-optim-pi

Conversation

@gitRasheed

Copy link
Copy Markdown
Collaborator

nns_arma_optim computed errors and the bias on the raw model with the selected weights, before the shrink and nns.regress decisions, so once either was taken, results came from a different model than errors and bias.shift described. The interval was results +/- (|UPM.VaR(errors)| + |bias|): symmetric, from the wrong model's errors, and widened by a bias already subtracted. The book bootstraps the forecast path and takes the lower bound from LPM.VaR (ch25:513-535, ch26:544, ch17:276). We mirror OVVO-Financial/NNS#88, which makes R follow the book. That PR changes the method, so the R author may close it; if so, this slice should be dropped too.

Stacked on #173; merge with a merge commit.

Changes

  • The bias step runs after shrink and nns.regress, on the validation forecast of the returned model, and is kept only if it improves the objective. We remove the bias blocks in the weights stage, whose result was always overwritten.
  • errors is that forecast minus actual, before the bias, so errors == results - bias.shift - actual on the validation window. On AirPassengers[:132] with training_set=105 and lags [6, 12, 24], objective "max", the max of that difference goes from 107.26 to 0.
  • The interval adds lpm_var/upm_var((1 - pred_int)/2, 0, residuals) of the validation residuals to the results. On the same case the band goes from results +/- 205.89 to +65.76 / -130.63, and on the two-cycle seed-33 series from +/- 6.35 to +4.12 / -5.85, with bias.shift going from 0.7387 to 0.

Tests

The four optim selection cases from #166 compare errors, results, bias.shift and the lower band again. Four more NNS.ARMA.optim.custom entries predate the R change and are refilled; the other five came back identical. The nine NNS.VAR tests that were strict xfails since #170 pass and lose their marks. The parity cache encodes R 13.3 plus the fixes in OVVO-Financial/NNS#73-#92.

Mirrors OVVO-Financial/NNS#88
Part of OVVO-Financial/NNS#71

Review

GPT-6.1 Sol reviewed this slice and raised one finding, which we rejected for now: validation residuals are taken before the zero clip, so the intervals describe a slightly different forecast when the bias-adjusted prediction goes negative. R (OVVO-Financial/NNS#88) does the same, and this slice matches it to 7 digits. The fix belongs in R first, then here.

…im-pi-from-returned-model)

Mirrors R's optim-pi-from-returned-model branch (NNS.ARMA.optim).
errors and the bias were computed on the raw model with the selected
weights, before the shrink and nns.regress decisions, so once either was
taken the results came from a different model than errors and
bias.shift described. The interval was results +/- (|UPM.VaR(errors)| +
|bias|): symmetric, from the wrong model's errors, and widened by a bias
already subtracted.

The bias step now runs after shrink and nns.regress, on the validation
forecast of the returned model, and is kept only if it improves the
objective. errors is that forecast - actual before the bias, so
errors == results - bias.shift - actual on the validation window. The
interval adds lpm_var/upm_var((1 - pred_int)/2, 0, residuals) of the
validation residuals actual - (forecast - bias) to the results, as the
book builds intervals (ch. 17, 25, 26). The bias blocks in the weights
stage are removed; their result was always overwritten.

Before -> after:
  AirPassengers[:132], training_set=105, seasonal_factor=[6, 12, 24], max
    max|errors - (results - bias.shift - actual)|   107.26 -> 0
    errors[:3]       19.40 71.65 114.35 -> 49.10 97.75 73.53
    band             results +/- 205.89 -> +65.76 / -130.63
  two-cycle series (seed 33), h=10: bias.shift 0.7387 -> 0,
    band results +/- 6.35 -> +4.12 / -5.85

Closes what the earlier optim and seasonality commits deferred: the four
optim selection cases compare errors, results, bias.shift and the lower
band again, and the nine NNS.VAR tests are no longer xfail. Four more
NNS.ARMA.optim.custom entries predate the R change and were refilled; the
other five came back identical.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head a48cdc33ecf59507caf3b9a63dbff6118fa05e52 is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/book-optim-pi branch November 7, 2026 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant