Point-in-Time Options Data: Why Most Options Backtests Are Optimistic

Point-in-time integrity means one thing: the value you read for a past timestamp is the value that would have been observable at that timestamp. Not corrected later, not computed with knowledge of what came next, not filtered by what survived.

It is one section of a wider vendor evaluation - the due-diligence checklist covers the rest - but it is the section that most often decides whether an archive is research-grade. Equity researchers have internalised this because Compustat restatements and index-membership survivorship taught the industry the hard way. Options research is younger, the archives are newer, and the same failure modes are less well policed. They are also easier to hit, because options data has more derived layers between the tape and the number you actually use.

Failure 1: restatement

Upstream data gets revised. Open interest is corrected, a bad print is removed, a settlement is amended. The question is what your archive does when the vendor re-pulls that day.

Most archives overwrite in place. The row for 2019-03-14 becomes whatever the latest pull says it is. Nothing is logged, nothing is versioned, and the value you backtested against last quarter may no longer exist.

The test: pull a fixed window today, store the extract, and pull the identical window again in a month. Diff them. Any difference tells you the archive restates. Then ask the vendor whether re-pulls are logged, and pin your reproducibility-critical research to your own frozen extract rather than to their live archive.

Failure 2: derived-statistic leakage

This is the one that most often survives review, because the raw data is clean and only the derived layer is contaminated.

Percentiles, z-scores, ranks and regime labels are computed over a window. If that window is the full dataset rather than the data available at each point, every historical value carries information from its own future. An IV rank computed against the whole year says "today is in the 90th percentile of a year you had not lived through yet".

The symptom is a strategy that trades regime boundaries beautifully in backtest and mediocrely in production, because in backtest it knew where the boundaries would be.

The test: take a derived statistic for a date early in the archive. Recompute it yourself using only data strictly prior to that timestamp. If the vendor's number and yours disagree systematically, the statistic is computed over the full sample. Ask directly: are percentiles and z-scores constructed using only data prior to their as-of timestamp? It is a yes-or-no question and a good vendor answers it in one word.

Failure 3: survivorship

Options add a wrinkle equities do not have: contracts expire by design. An archive that prunes expired contracts to save space cannot answer what the chain looked like on a past date, only what remains of it. Delisted underlyings compound the problem - a universe study that silently excludes companies that went away is measuring the survivors.

Note the distinction that trips people up. Retained means expired contracts and delisted names are still queryable. Survivorship-adjusted means the universe reflects historical membership rather than today's. Retention is necessary; it is not sufficient, and vendors sometimes claim the first while implying the second.

The test: query a chain for a date more than a year back and check that contracts which have since expired are present with sane open interest. Then pick an underlying that was delisted or acquired and confirm it returns data for dates when it traded. Then ask specifically whether index or universe membership is historically accurate, because that is a different question with a different answer.

Failure 4: live-versus-historical drift

The subtlest failure, and the one unique to computed analytics rather than raw data.

Suppose the live endpoint and the historical endpoint are separate systems, as they usually are, because live optimises for latency and historical for range queries. If they compute the same metric with even slightly different code - a different filter threshold, a different rounding convention, a different treatment of thin strikes - then your backtest validates one thing and production runs another. Nothing errors. The two just quietly disagree, and the disagreement is invisible because you never see them side by side.

The test: take a timestamp from earlier today, after the live value has been captured. Query the live endpoint's stored value and the historical endpoint for that same timestamp. They should match to the precision you care about. Do this for a few symbols across a few metrics. Then ask the vendor whether an automated test asserts parity between the two engines, and whether it runs on every change or was performed once.

What an honest answer sounds like

Not "our data is point-in-time accurate". That phrase is marketing and covers all four failures without addressing any. An honest answer is specific and usually mixed, because no archive is perfect on every axis:

History is retained as last pulled and is not restatement-versioned - a re-pull overwrites in place, so pin reproducibility-critical work to a fixed extract. Expired contracts and delisted underlyings are retained and queryable, but the universe is not membership-adjusted in the index sense. Derived percentiles and z-scores are leak-bounded by construction. Live and historical share response shapes and are held in numerical parity by an automated cross-engine test.

That answer admits two limitations and claims two guarantees. It is more useful than an unqualified assurance, because you can design around stated limitations and you cannot design around a claim that turns out to be approximate.

Where FlashAlpha stands on each

Stated plainly, and in the same words we use in the methodology whitepaper rather than a softened version for an article:

• Restatement: not versioned. The historical store is as last pulled. A day already loaded is skipped on re-ingest, and a forced re-pull overwrites in place. Reproducibility-critical work should pin to a fixed extract.
• Derived statistics: leak-bounded. Percentiles and z-scores are constructed using only data strictly prior to their as-of timestamp. This is the one axis guaranteed by design.
• Survivorship: retained, not adjusted. Rows are never deleted on expiry or delisting, so expired contracts and delisted underlyings remain queryable for any past timestamp. The dataset is not survivorship-adjusted in the index-membership sense, and cross-sectional studies must account for that.
• Live versus historical: same contract, parity-tested. Every core analytic has a historical twin with an identical response shape via an at= selector, and the live path and the historical hydrator are held numerically in parity by an automated cross-engine test.
• Known gaps are published. In the minute-resolution store, per-contract bid and ask sizes and volume are not populated, intraday smoothed IV is gated null in backtest mode, and a per-symbol coverage report exposes missing sessions rather than hiding them.

Two of those are limitations. We publish them because a researcher who discovers a limitation after building on it loses more than the limitation was ever worth.

If the archive fails these tests badly enough, the alternative is owning the pipeline yourself, which has its own price - see build vs buy. None of these tests take longer than an afternoon, and all four are worth running before a strategy depends on the archive rather than after. The result is rarely a clean pass or fail - it is a map of which guarantees hold and which limitations you need to design around, which is exactly what a research process should produce. FlashAlpha's positions on all four are published in the methodology whitepaper, including the two axes where we state a limitation rather than a guarantee.