Point-in-Time Options Data: Why Most Options Backtests Are Optimistic

Point-in-time integrity means one thing: the value you read for a past timestamp is the value that would have been observable at that timestamp. Not corrected later, not computed with knowledge of what came next, not filtered by what survived.

It is one section of a wider vendor evaluation - the due-diligence checklist covers the rest - but it is the section that most often decides whether an archive is research-grade. Equity researchers have internalised this because Compustat restatements and index-membership survivorship taught the industry the hard way. Options research is younger, the archives are newer, and the same failure modes are less well policed. They are also easier to hit, because options data has more derived layers between the tape and the number you actually use.

Failure 1: restatement

Upstream data gets revised. Open interest is corrected, a bad print is removed, a settlement is amended. The question is what your archive does when the vendor re-pulls that day.

Most archives overwrite in place. The row for 2019-03-14 becomes whatever the latest pull says it is. Nothing is logged, nothing is versioned, and the value you backtested against last quarter may no longer exist.

The test: pull a fixed window today, store the extract, and pull the identical window again in a month. Diff them. Any difference tells you the archive restates. Then ask the vendor whether re-pulls are logged, and pin your reproducibility-critical research to your own frozen extract rather than to their live archive.

Failure 2: derived-statistic leakage

This is the one that most often survives review, because the raw data is clean and only the derived layer is contaminated.

Percentiles, z-scores, ranks and regime labels are computed over a window. If that window is the full dataset rather than the data available at each point, every historical value carries information from its own future. An IV rank computed against the whole year says "today is in the 90th percentile of a year you had not lived through yet".

The symptom is a strategy that trades regime boundaries beautifully in backtest and mediocrely in production, because in backtest it knew where the boundaries would be.

The test: take a derived statistic for a date early in the archive. Recompute it yourself using only data strictly prior to that timestamp. If the vendor's number and yours disagree systematically, the statistic is computed over the full sample. Ask directly: are percentiles and z-scores constructed using only data prior to their as-of timestamp? It is a yes-or-no question and a good vendor answers it in one word.

Failure 3: survivorship

Options add a wrinkle equities do not have: contracts expire by design. An archive that prunes expired contracts to save space cannot answer what the chain looked like on a past date, only what remains of it. Delisted underlyings compound the problem - a universe study that silently excludes companies that went away is measuring the survivors.

Note the distinction that trips people up. Retained means expired contracts and delisted names are still queryable. Survivorship-adjusted means the universe reflects historical membership rather than today's. Retention is necessary; it is not sufficient, and vendors sometimes claim the first while implying the second.

The test: query a chain for a date more than a year back and check that contracts which have since expired are present with sane open interest. Then pick an underlying that was delisted or acquired and confirm it returns data for dates when it traded. Then ask specifically whether index or universe membership is historically accurate, because that is a different question with a different answer.

Failure 4: live-versus-historical drift

The subtlest failure, and the one unique to computed analytics rather than raw data.

Suppose the live endpoint and the historical endpoint are separate systems, as they usually are, because live optimises for latency and historical for range queries. If they compute the same metric with even slightly different code - a different filter threshold, a different rounding convention, a different treatment of thin strikes - then your backtest validates one thing and production runs another. Nothing errors. The two just quietly disagree, and the disagreement is invisible because you never see them side by side.

The test: take a timestamp from earlier today, after the live value has been captured. Query the live endpoint's stored value and the historical endpoint for that same timestamp. They should match to the precision you care about. Do this for a few symbols across a few metrics. Then ask the vendor whether an automated test asserts parity between the two engines, and whether it runs on every change or was performed once.

What an honest answer sounds like

Not "our data is point-in-time accurate". That phrase is marketing and covers all four failures without addressing any. An honest answer is specific, one line per axis:

Point-in-time inputs are never restated - replay runs today's engine over the exact information set known at that minute, and every response carries data_as_of, archive_as_of and endpoint_version. Expired contracts and delisted underlyings are retained and queryable for any past timestamp. Derived percentiles and z-scores are leak-bounded by construction. Live and historical share response shapes and are held in numerical parity by an automated cross-engine test.

That answer is specific on every axis. It is more useful than an unqualified assurance, because you can design around a stated guarantee and you cannot design around a claim that turns out to be approximate.

Where FlashAlpha stands on each

Stated plainly, in the same words we use in the methodology whitepaper:

• Restatement: inputs are never restated. Replay runs today's engine over the exact information set known at that minute, and every response carries data_as_of, archive_as_of and endpoint_version, so any run is traceable to the engine and the information set that produced it.
• Derived statistics: leak-bounded. Percentiles and z-scores are constructed using only data strictly prior to their as-of timestamp. By design.
• Survivorship: retained. Rows are never deleted on expiry or delisting, so expired contracts and delisted underlyings remain queryable for any past timestamp. Index membership is a list you supply, so a cross-sectional study can use the membership as of each rebalance date.
• Live versus historical: same contract, parity-tested. 50+ live routes have a historical twin on an identical path with an identical response shape via an at= selector, and the live path and the historical hydrator are held numerically in parity by an automated cross-engine test.
• Coverage is self-describing. GET /v1/tickers returns the loaded symbols and date ranges - 200+ symbols, 54 with 8+ years of minute-level history, 14 from January 3, 2017 - so a study can be sized against the archive before the first call.

We publish all of them in the same words because a researcher can only put weight on a guarantee that is stated precisely.

If the archive fails these tests badly enough, the alternative is owning the pipeline yourself, which has its own price - see build vs buy. None of these tests take longer than an afternoon, and all four are worth running before a strategy depends on the archive rather than after. The result is a map of which guarantees hold, which is exactly what a research process should produce. FlashAlpha's positions on all four are published in the methodology whitepaper.