The earlier numerical ranking on this page is withdrawn. Its NNS call
included scored outcomes in the optimizer's internal validation data,
while the other methods used observed one-step lags and NNS forecast a
longer chunk from one origin. Those scores cannot support a comparative
performance or coverage claim. The original material remains available
in Git history at commit
1075a338d932db13a8210e8ed23fb94f70a3f21f.
The executable source is nns_arma_interval_benchmark.R. It retains the simulated nonlinear heteroskedastic process and the existing comparison methods, with these corrections:
MAX_H is one and any different value is
rejected.h = 1. Seasonal candidates are selected
from the earlier internal training prefix. The remaining observed prefix
is internal validation; the next outcome is scored only after
forecasting.NA for NNS, as for other interval-only methods. A
degree-two partial-moment transform of three supplied points would
define an additional modelling choice, not a native predictive density
or a validated probability calibration.sessionInfo(), seeds, and optional
dependency availability with results.source("examples/nns_arma_interval_benchmark.R")
result <- run_all()
sessionInfo()The full five-seed, 3,500-point experiment now entails thousands of honest one-step optimizer fits and can take substantial time. The corrected source is provided for that experiment; this page does not substitute a small smoke run for a full benchmark or carry forward the invalid historical numbers.
The comparison estimates performance for this process, seed set, horizon, training policy, and loss. Report uncertainty across series or origins and inspect coverage by volatility regime. The true conditional oracle is a reference with privileged information, not an available forecasting method. Conformal finite-sample guarantees require the applicable exchangeability or other calibration assumptions; this serially dependent, changing process does not satisfy ordinary iid split-conformal assumptions automatically. NNS residual bands also require empirical out-of-sample validation.
The baseline uses glmnet ridge when installed and
lm otherwise; these are separate configurations and must be
identified in results. Model refitting policies are likewise part of the
compared procedure: NNS updates its fit at origins while the original
lagged baseline is fitted on the initial training window. A claim about
estimator quality alone requires an additional matched refitting-policy
comparison.
See example corrections for the related distinction between frequency calibration and directional severity thresholds.