# Audit and reproduction guide — revision 1.1 ## Reviewer quick start Unzip the package and enter `options-research-taster`. Use CPython **3.13.14**, the recorded reference runtime; only the standard library is required. Other Python versions may reproduce the values but are not certified to produce identical bytes. No server credentials, network connection, package install or FlashAlpha application build is needed. ```text python audit.py check python audit.py reproduce ``` `check` validates every frozen input and output against `release.json`. `reproduce` first performs that check, copies **only raw data, source code and static protocol/documentation** into fresh temporary directories, and rebuilds all 24 generated outputs twice, with different Python hash seeds. It requires every regenerated file to match the sealed SHA-256 exactly. Socket creation and child processes are blocked inside pipeline steps. Existing CSV results are not copied into the rebuild directories. The result and complete per-command logs are written to `reproduction_report.json`. Temporary directories are owned by the runner and removed after each run. The report records runtime, operating system and release hash. It is an execution record, not an independent reviewer signature. **Do not run `seal` to make a failing reproduction pass.** Preserve the failed evidence, investigate the difference, and issue a new revision when changing inputs or methods. `seal` is an explicit maintainer command; reproduction never calls it. ## What is frozen `release.json` fingerprints every analysis input, script, protocol, audit instructions and generated output. `protocol.json` holds the existing numerical choices; `research.py` holds the exact 22 leg constructions, six headline profiles, formula implementations and filter order. `source_index.csv` lists every raw extract's SQL, row count, retrieval time, SHA-256 and role. The rejected dividend-helper price source is retained and clearly marked as unused. Original database responses, including column schemas, are in `raw/`. The extract is a saved snapshot; re-querying today's archive is a **new extraction** and may differ after upstream corrections. The pipeline rejects a cached response when its SQL differs from the requested query. The original v1 ZIP was preserved separately as `options-research-taster-v1-original.zip`. Revision 1.1 strengthens audit mechanics and wording. The 22-profile economic results and six-profile ranking are intended to remain unchanged; the original archive allows that comparison. ## Follow a claim all the way to its quotes 1. Open `quote_bank.json` or the article table and identify the strategy. 2. `claim_audit.json` contains its exact headline metrics, predicate and complete list of included `trade_id` values. `cohort_membership.csv` explains whether each of the 216 planned ETF-month windows enters the shared cohort and each headline strategy's exclusion reason. 3. Join a `trade_id` to `trades.csv` and `legs.csv`. The dollar P&L, multiplier, fees, quantities, dates and normalization denominator are explicit. 4. `quote_lineage.csv` identifies the **raw filename and zero-based JSON array row** for each option signal/entry/exit quote and underlying entry/exit quote. For example, `/dataset/123` means element 123 of the response's `dataset` array; interpret columns with the same file's `columns` schema. 5. Verify the raw file hash against `source_index.csv` and the sealed `release.json`. `selection_ledger.csv` includes every profile/role for every planned window, including excluded ones. It records chosen contracts where selection ran, quantities and the first exclusion reason. All candidate chains remain in raw files, so a reviewer can inspect why a different contract was not selected. The ledger is a replay of the selection code, not a separately designed selection algorithm. An excluded row contains the **first** failing reason, not all possible reasons. Two windows with invalid underlying quotes never reach option selection. A missing role can mean no eligible contract or unavailable Greeks; detailed candidates are in the corresponding raw chain. This limitation is explicit rather than represented as a fictitious trade. ## Independent calculation and adversarial checks `audit_tables.py` recomputes every completed trade from original raw quotes using `Decimal` and separate cash-in/cash-out accounting. It does not use the stored leg P&L as its input. It also recomputes every headline mean, win rate and execution drag from the reconciled trades. The maximum observed reconciliation difference is recorded in `claim_audit.json`. The existing `research.py verify` checks chronology, source quote matches, cash flows, notional normalization, opposite-structure symmetry, source hashes and common-cohort rankings. `test_research.py` tests crossed/invalid quotes, strike/expiry tie breaking, missing deltas, refusal to substitute a later eligible contract, changed or missing input rejection, and stale SQL cache rejection. The clean builds run those tests with the rest of the pipeline. Do not use `python -O`: the older verifier relies on assertions. The article table is generated from the results. Some v1 narrative statements remain version-specific prose. `publication_checks.json` lists explicit guards for their numerical content, evaluated against the reconciled claim audit and frozen protocol. If a guarded number changes, article generation fails and requires editorial review. This prevents silent stale statistics; it does not convert interpretive prose into a statistical proof. ## Reproduction boundary and remaining scientific limitations - **Reproduced:** raw archive responses -> schedule -> contract selection -> exclusions -> quote-based P&L -> summaries -> article table and guarded numerical claims. - **Not reconstructed:** original vendor capture, exchange book state, archived Greek calculations/model version, each EOD contract's last quote timestamp, live fill/queue behavior, corporate-action announcement availability or early assignments. - The raw chain extract restricts strikes to **75–125% of signal spot** and expirations to **25–80 days after the signal date**. Published choices are made within that saved universe. This is not the entire historical options market. - Source hashes show that saved bytes agree with a trusted release fingerprint; an unsigned hash is not proof the vendor data was originally correct. Keep the ZIP checksum in a trusted external location or public release record. A person who changes both files and their checksum can create a different package. - The 133 headline observations come from selected complete cases across correlated ETFs. There are 65 calendar months. No full quote-freshness audit, missingness correction, equal-risk comparison or untouched out-of-sample validation was performed. - The three timestamp checks are deterministic spot checks only. An initial broad query hit its time limit; that limitation remains in `timestamp_spotcheck.json`. - Confidence intervals are exactly reproducible outputs of the documented exploratory bootstrap. No independent statistical-method review has certified its assumptions. - Calendar dates come from observed source days. The monthly schedules agree across the ETFs but were not certified against an independent exchange calendar. - Short-option early assignment, collateral yield and full trade-path risk are not modeled. Reproducibility does not turn the sample into evidence of executable alpha. - This review and its second arithmetic implementation were produced in the same assistant workflow. **An external human/statistical review is still outstanding.** No approval is implied by a passing script. ## External reviewer checklist - [ ] Record and externally anchor ZIP SHA-256; run `check` and `reproduce`. - [ ] Inspect five randomly chosen trades, one losing trade and one exclusion through the raw-row pointers. - [ ] Review selection rules and the strike/expiry extraction bounds. - [ ] Assess whether the common notional denominator answers the article's question. - [ ] Assess exclusions, dividend handling, missing quotes and correlated observations. - [ ] Review bootstrap assumptions and whether the wording overstates evidence. - [ ] Record reviewer identity, date, release hash and findings before public release. ## Maintainer build and package commands These commands create a new audited release; reviewers should use `check` / `reproduce` instead. ```text python audit.py build python audit.py seal python audit.py reproduce python audit.py package python audit.py archive --zip ../options-research-taster.zip ``` `package` writes the complete ZIP and adjacent `.zip.sha256`. The final command extracts that exact ZIP, runs the offline reproduction again, and writes `options-research-taster-archive-audit.json` beside it, including the tested archive hash. This final report is outside the archive so adding a report does not change the artifact it certifies. Nothing is deployed or publicly published by these commands.