Data Quality & Monitoring

Abstract

Every data vendor says their data is good. This page documents how FlashAlpha verifies it: a five-layer architecture in which chains are reconciled against an independent second source contract by contract, serving endpoints refuse with machine-readable errors rather than degrade, every pipeline reports into a central health registry, and the historical archive is never silently restated. The design principle throughout is fail closed and say why - a wrong answer is worse than a refused one.

In the spirit of the methodology document, this page describes capabilities and guarantees at the level a due-diligence reviewer needs. It deliberately does not publish operational parameters - tolerances, schedules, source pairings, or alert routing - that would constitute either a turnkey clone of the monitoring stack or a map for gaming it.

The five layers

Layer 1 - Ingest integrity

Market data arrives from exchange-grade feeds with instrument-mapping handled before anything is stored, so a contract's identity is stable through symbol reuse and rolls. Open-interest snapshots are written idempotently - re-running a snapshot can never double-count - and expired contracts are evicted on schedule so chains never carry dead weight. Where a feed's own semantics create known edge cases (the daily OI publication burst, session reconnects), the ingest layer is built around those semantics rather than around hope.

Layer 2 - Cross-source reconciliation

The core discipline: check the data against sources that have no reason to agree.

• Chain reconciliation. Option chains are compared contract by contract against an independent second source - coverage, open interest, and price agreement scored per symbol, tens of thousands of contract-level comparisons per sweep.
• Parity re-derivation. Index spot is recovered from the option chain itself via put-call parity and compared with the vendor print; futures are anchored against cash. A quote stream can drift; an arbitrage identity cannot.
• Reference-data cross-checks. Earnings dates are verified across two independent providers daily, with drift surfaced for review before it can contaminate event analytics.
• Conservation checks. Aggregates that must reconcile with their parts (served open interest against the underlying chain) are checked as invariants, not assumed.

Layer 3 - Serving guards: fail closed

Verification would be theatre if degraded data were served anyway. The serving layer refuses instead, with machine-readable faults a pipeline can branch on:

• A price series that is not fresh enough to model on is refused with HTTP 409 and a stale_data fault - including the date of the last good bar and its staleness in trading days - rather than serving a frozen price:

HTTP/1.1 409 Conflict
{
  "problem": "stale_data",
  "message": "The historical price archive's most recent bar is 2026-07-28
              (5 trading days old). Data is stale; refusing to serve a frozen
              price or fit models on it.",
  "dataAsOf": "2026-07-28",
  "staleTradingDays": 5
}

• One-sided and ghost quotes (a bid with no ask, weekend residue) are excluded from IV solving rather than solved into a fictitious surface.
• The cross-sectional screener refuses to rank when source coverage falls below what a fair ranking requires.
• A fleet-wide availability control can withdraw serving entirely - the platform prefers absence over wrongness at every scale.

Layer 4 - Fleet & job health

Every pipeline in the platform - ingest jobs, reconciliation monitors, serving processes, across every host in the fleet - reports into a central job-health registry with status, last-run time, staleness, and consecutive-failure tracking. The registry is watched continuously, and a daily digest reaches humans whether or not anything is wrong: silence is never assumed to mean health. Independently of all of this, external probes exercise the platform from outside its own infrastructure, and availability is reported publicly at /status.

Layer 5 - Point-in-time integrity

Stored inputs are point-in-time and never restated: what replay sees at any minute is what a live consumer could have known at that minute, and corrections are documented in the changelog, never applied invisibly under a backtest. Analytics recompute through the same engine that serves live traffic, with live-historical parity pinned by test - the full computation model is documented at Historical Data & Computation. This is why the guards in layer 3 exist: a value that fails verification must be refused now rather than quietly fixed later.

Every response carries its own telemetry

Rather than ask consumers to trust a snapshot blindly, the API attaches machine-readable quality signals on the 0DTE and screener paths that a pipeline can gate on: a composite data-quality score built from valid-IV share, valid-Greek share and snapshot freshness; a Greek-smoothness score that penalises jagged, thin chains; and the snapshot age and contract count behind every aggregate. The exact score construction is published in methodology §7. If you would rather gate on our numbers than trust them, that is exactly the intended use.

Independent verification

The monitoring stack itself is under test - reconciliation, parity validation, staleness monitors, the job-health registry and the serving guards all carry dedicated automated suites, and engine parity is pinned by fixture. Uptime is probed from outside the platform's own infrastructure and published at /status. For security posture and operational due diligence, see /security; for licensing and firm-wide terms, /institutional.

What we publish, and what we share privately

This page is deliberately written at the level of architecture and guarantees. The operational parameters underneath it - reconciliation tolerances, sweep schedules, source pairings, alert routing, and escalation rules - are withheld from the public page as a matter of policy: publishing them would hand a build checklist to imitators and a timing map to anyone probing the platform, while adding nothing to the guarantee itself.

Withheld does not mean unavailable. Prospective enterprise and systematic customers can review parameter-level detail - including the monitoring configuration, incident and escalation practice, and SLA riders scoped to the engagement - as part of commercial due diligence under NDA. Start at /institutional, or the security & due-diligence page for the operational-posture questionnaire answers. The distinction to hold us to: everything verifiable from outside is public (this page, /status, the per-response telemetry, the live counters above); everything that only matters to an operator or an attacker is not.

Frequently asked questions

How does FlashAlpha know its options data is right?

By checking it against sources that have no reason to agree: contract-level reconciliation against an independent second source, index spot re-derived from the chain via put-call parity, futures anchored to cash, and earnings dates cross-verified across two providers. Disagreement raises an alert; it is never papered over.

What happens when a data source degrades?

The API refuses rather than degrades: 409 stale_data instead of a frozen price, ghost quotes excluded from IV solving, the screener declining to rank on thin coverage. Failure modes are explicit and machine-readable, never silent.

Do you restate historical data?

No silent restatement, ever. The archive serves what was computed at the time; corrections are documented in the changelog. That property is what makes point-in-time backtesting on this platform valid.

How is data quality monitored across the fleet?

Every pipeline reports into one job-health registry - status, staleness, consecutive failures - watched continuously and summarized to humans daily, with external probes and a public status page verifying the platform from outside.

Can we review the monitoring configuration during due diligence?

Yes. Parameter-level detail - tolerances, schedules, source pairings, escalation practice, and SLA riders - is shared with prospective enterprise and systematic customers under NDA as part of commercial due diligence, via /institutional. The public page stays at the architecture level by policy.