If you searched for walk-forward optimization or walk-forward validation for trading signals, you have probably seen the mechanical recipe. This page adds the two parts the recipe usually omits - purging/embargo for overlapping outcomes, and out-of-sample controls done without leakage - and runs the whole thing on a real, public, 1,971-session options dataset so you can check every number.
11/11
OOS windows where raw GEX vs next-day RV stayed negative
-0.31
Mean out-of-sample raw rank IC
-0.05
Mean OOS IC after VIX + ATM IV controls (train-fitted)
4
Sign flips of the rolling controlled beta in 8 years
Why k-fold is the wrong tool for market signals
Standard cross-validation shuffles observations into folds as if they were independent. Options-signal observations are not independent in two specific ways:
- Serial information. Training on 2023 and testing on 2021 lets the model "remember the future" through any persistent regime variable. Validation has to move forward in time only.
- Overlapping outcomes. If the outcome is 5-day forward realized volatility, the observation dated Monday and the one dated Tuesday share four days of outcome window. Split them across train/test and the test set contains information the train set already saw. The fix is mechanical: purge the sessions whose outcome windows straddle the split, and add an embargo - a buffer of sessions after the train end that goes to neither side.
Our worked example predicts next-day RV, so the overlap is small - but we run a 5-session embargo anyway, because the cost is 5 rows and the habit is the point. Predicting 5-day or 20-day outcomes without purging is one of the quiet ways backtests inflate; it belongs on the same checklist as the fill model (see transaction cost models).
The worked example
Setup, fixed before results were computed. Data: the public panel from our 8-year GEX/DEX/VEX/CHEX backtest - 1,971 usable SPY sessions, 2018-04-16 to 2026-04-01 (master_dataset.csv, unchanged). Signal: net GEX level. Outcome: next-day realized volatility. Scheme:
- Expanding train window, minimum 504 sessions (~2 trading years).
- 5-session embargo, then a 126-session (~6 month) test window. Repeat, stepping forward 126 sessions - 11 windows ending 2020-2025.
- Per window we compute four numbers: train and test raw Spearman IC, and train and test partial IC after residualizing both signal and outcome on VIX and ATM IV. The control betas are fitted on the train window only and applied unchanged to the test window - refitting controls on test data is leakage wearing a suit.
Results: the signal a live researcher would have found
| Train end | Test window | Train raw IC | OOS raw IC | Train partial IC | OOS partial IC |
|---|
| 2020-04-15 | 2020-04-23 to 2020-10-20 | -0.437 | -0.349 | 0.003 | 0.048 |
| 2020-10-13 | 2020-10-21 to 2021-04-22 | -0.428 | -0.419 | 0.009 | -0.156 |
| 2021-04-15 | 2021-04-23 to 2021-10-20 | -0.417 | -0.396 | -0.014 | -0.027 |
| 2021-10-13 | 2021-10-21 to 2022-04-21 | -0.425 | -0.373 | -0.025 | 0.034 |
| 2022-04-13 | 2022-04-22 to 2022-10-20 | -0.430 | -0.100 | -0.031 | 0.144 |
| 2022-10-13 | 2022-10-21 to 2023-04-24 | -0.432 | -0.200 | -0.010 | -0.181 |
| 2023-04-17 | 2023-04-25 to 2023-10-23 | -0.413 | -0.138 | -0.006 | 0.058 |
| 2023-10-16 | 2023-10-24 to 2024-04-24 | -0.364 | -0.365 | -0.007 | -0.168 |
| 2024-04-17 | 2024-04-25 to 2024-10-23 | -0.367 | -0.389 | -0.021 | -0.135 |
| 2024-10-16 | 2024-10-24 to 2025-04-28 | -0.364 | -0.337 | -0.032 | 0.091 |
| 2025-04-21 | 2025-04-29 to 2025-10-27 | -0.361 | -0.323 | -0.026 | -0.283 |
Read the two OOS columns side by side and the story writes itself:
- The raw effect is real and boringly stable. Negative in 11 of 11 out-of-sample windows, mean -0.308, never weaker than -0.10 even through the 2022 bear market. More positive gamma exposure is followed by lower next-day realized volatility, every window, no exceptions.
- The incremental effect is noise. After controlling for what a trader can already see on a VIX terminal, the mean OOS IC is -0.052, the sign is stable in only 6 of 11 windows, and the largest single-window values (+0.144, -0.283) are both bigger than the mean in absolute terms - the signature of noise, not signal. Walk-forward reaches the same verdict the full-sample study reached, which is exactly what should happen when the original analysis was honest.
Parameter stability: the overfitting test people skip
A signal you would actually size needs a coefficient that stays put. We refit the standardized regression rv_next_1d ~ net_gex + vix + atm_iv on a rolling 504-session window, stepping every 21 sessions - 70 fits across the sample (param_stability.csv). The standardized net-GEX beta ranges from -0.186 to +0.005 and crosses zero 4 times. A coefficient that spends eight years wandering between "small negative" and "nothing" is not a parameter you size a book on - it is a residual you monitor.
The checklist
- Validation moves forward in time only; no shuffled folds.
- Purge sessions whose outcome windows straddle the split; embargo a buffer after the train end. Cost: a handful of rows. Skipping it: silent leakage.
- Fit every control, scaler, and hyperparameter on the train window; apply frozen to test. If any statistic in the OOS column touched test data during fitting, it is not OOS.
- Report per-window results, not just the pooled number - 11 honest windows tell you more than one blended IC.
- Run rolling-coefficient stability. Signals with wandering betas fail live even when pooled ICs look fine.
- Pair with an honest fill model before any P&L claim - see the fill-model reference and alpha decay in options signals.
Reproduce it
Inputs and outputs are public: the panel, wf_windows.csv, param_stability.csv, and artifacts.zip with the ~90-line script. To run the same exercise on your own signals, the panel regenerates from the Historical API (Alpha tier) - every feature in it is one replayed endpoint call per session, and the same code path serves live values in production.
Walk-forward validation did not rescue a dead signal or kill a live one here - it confirmed, out of sample and without leakage, exactly what the full-sample study claimed: raw GEX carries real information about next-day volatility, and its increment over VIX and ATM IV does not. That is what validation is for. The scheme costs ~90 lines of code on top of data you can already replay point-in-time; there is no excuse to publish an options signal without it. Related: the original 8-year backtest, the complete quant backtesting guide, and leak-free VRP percentiles.
