Audit 03 — arithmetic and claim integrity of the historical evidence ledger

Scope. Independent, read-only audit of docs/research/history.md (4,607 lines, read in full), docs/research/status.md, docs/research/experiment-index.md, and docs/strategies.md, cross-checked against src/core/native/engine.hpp, research/schemas/game-result-v1.schema.json, research/benchmarks/baselines-v1.json, artifacts/protocols/optimistic-phase-ntuple/protocol.json, research/seeds/, and the approaches/ source tree. No file other than this one was modified. No gameplay, training, or benchmark was run. No cohort was opened.

Status of this document. Exploratory audit finding. It changes no frozen protocol, no result record, and no shared status table. Every recommendation in section (d) is a proposal for the coordinator, not an applied edit.

0. Method: the score identity used throughout

From src/core/native/engine.hpp:

so a wave of depth d pays 7 / 39 / 109 / 224 / 391 / 617 per popped disc for d = 1..6. A rise is awarded only when moves_remaining reaches zero and raiseCoveredRow succeeds (playMove, lines 297–322); a failed rise ends the game with no award. Therefore, for any complete game,

score = rises * levelBonus + clears * 70,000 + sum(waves)
moves/5 - 1  <=  rises  <=  floor(moves/5)

Every reported (mean score, mean moves) pair was tested against both candidate level bonuses. A mode is feasible only if the residual score - rises*bonus is non-negative at the upper rise bound and does not imply an absurd chain rate (> ~900 points/move) at the lower bound.

The test is fully discriminating. Because 17,000/5 = 3,400 and 7,000/5 = 1,400 points per move, the two feasible residual bands never overlap for any row in the ledger. All 203 rows that record a move count classify into exactly one mode. Two rows record no move count and are unclassifiable.

Calibration of the rescoring constant. The "Corrected Hardcore scoring replay" (history.md:3327–3347) is a deterministic replay of the same already-consumed seeds. Its move counts are byte-identical to the original 7,000-point run, and the score differences are exact multiples of 10,000:

This proves two things the ledger never states explicitly:

1. The level bonus does not change fair-expectimax decisions. Identical trajectories, identical move counts, identical clear/reveal counts. The engine's level award is added after the search has chosen.
2. Any 7,000-point result in the ledger can be exactly rescored as score_17k = score_7k + 10,000 * rises, with rises ≈ moves/5 − 0.925 calibrated from the four rows above (equivalently Δ ≈ 2,000·moves − 9,250).

Point 2 is what makes several of the findings below actionable rather than rhetorical: three preregistered gate outcomes can be recomputed, and two of them reverse.

(a) Classification of every reported (score, moves, n) triple

205 rows. "Implied non-level pts/move" is the min to max range of non-level points per move (chain waves plus any undisclosed board clears) implied by the inferred mode. The minimum is evaluated at the upper rise bound floor(moves/5) and is routinely slightly negative, because real games almost always end on a failed rise, putting true mean rises near moves/5 − 0.925 (calibrated in section 0). The operative figure is therefore the maximum, which is the residual at the lower rise bound. "Mode stated in ledger?" is whether the surrounding history.md section names the scoring mode at all.

Summary of section (a)

Positive finding. The ledger's numbers are arithmetically sound. Not one (score, moves) pair is impossible under both bonuses, and not one is compatible with both. There is no detectable transcription error in 205 reported results. Independent corroboration: 26 approach sources carry a kLevelBonus == 7'000 build assertion, and every one of them matches a row this audit classified as 7k from the arithmetic alone.

Negative finding. Of the 119 rows that are 7,000-point Sequence-scored, only 38 sit inside a history.md section that says so. 81 rows across 23 experiment sections carry 7,000-point scores with no scoring-mode label at all. The ledger's own disclaimer (history.md:57, repeated at 3347) scopes itself to "experiments below that identify levelBonus: 7000" — and grep finds that string in exactly two places in the file, both of which are the disclaimer itself. The disclaimer's selector matches zero experiment sections.

The unlabeled 7k sections are:

accessible-energy-root-prior (773) · transition-reward-horizon (1091–1092) · fair-cem-optimizer (1150–1159) · fair-cem-depth4-interaction (1220–1229) · fair-depth4-s7 (1288) · oracle-topology-residual-extension (1452–1465) · fair-d1-rollout-improvement (1563–1564) · ppo.hpp v1 audit (1611–1613) · ppo-v2 (1633–1634) · fair-phase-energy-release (1688–1700) · fair-clear-reward-confirmation (1751–1752) · fair-selective-depth (1819–1829) · fair-cycle-boundary-depth5 (1886) · fair-depth5-s3 (1949–1950) · d4-phase5-value-veto (2003) · scaled-d4-distill D2 diagnostic (2103–2104) · fair-reveal-reward (2167–2189) · d4-d2-rollout-veto (2255–2256) · d4-d2-rollout-veto-quality-extension (2764–2768) · d2-vertical-ladder-probe (2830–2831) · fair-vertical-ladder-depth4 (2845–2846) · ntuple-tc (3092–3099) · ntuple-phase-conditioned (3178).

This is not cosmetic. It means the entire fair-D4 ablation programme — seven-stratum chance, CEM-tuned leaf coefficients, selective D5, cycle-boundary D5, full-width D5/s3, clear reward, reveal reward, vertical-ladder energy, phase-energy release, the 25-move rollout veto, and the phase-5 value veto — was conducted, gated, and rejected under the wrong level bonus, and history.md does not say so at any of those sections.

Two board-clear observations

The 70,000-point board-clear bonus is never counted anywhere in the repository. research/schemas/game-result-v1.schema.json has no boardClears field, no rowRises field, and no scoringMode/levelBonus field; neither does the "Standard per-game record" list in docs/benchmarks.md. The residual column in section (a) therefore cannot be decomposed into chains versus clears for any run.

The residuals nevertheless carry a real signal that the ledger never states. Public fair searches sit at 120–230 non-level points per move. The privileged future-aware oracle sits at 730–830 (history.md:568, 607, 722, 735) — a 4–6× higher chain-scoring density, not merely a longer game. That is a quantitative restatement of the "oracle gap" that is more informative than the raw score comparison, and it is currently invisible because no board-clear or per-depth wave counter is retained.

(b) Seed-range import table

research/seeds/ contains only a README that says "Do not assign a new range until historical usage has been conservatively imported." docs/benchmarks.md repeats the requirement. No registry exists. The table below is the conservative import, assembled from history.md, the frozen protocols, and hard-coded constants in approaches/.

Legend: CONSUMED = the ledger records the range as read; RESERVED = the ledger explicitly records it as declared-but-unopened; UNDOCUMENTED = the constant exists in source with no ledger entry.

Parity, self-test and calibration families

0x3d1–0x3d6 — training and fitting lanes

0x3d7… — the collision zone

The n-tuple baseline consumes consecutive seeds (src/core/native/ntuple.hpp:559, training_seed_start + game), so:

Twelve later ranges that the ledger or source calls "fresh", "training-only", or "collection" fall inside the first 100,000-game training block:

The 0x3d7a…–0x3d7f… oracle-distillation ranges (history.md:603–616) end above 0x3d77a11f and do not collide.

0x3d9–0x3df — development and fitting cohorts

0x3e… — fresh screens and gated confirmations (40 declared subranges)

No two 0x3e… subranges collide. 0x3ea90000–007 / 0x3eaa0000–00f were declared unread by oracle-topology-residual (history.md:1375) and later legitimately consumed by oracle-topology-residual-extension (history.md:1451, 1457); the import table must record them as CONSUMED.

0x3f…, 0x4d…, 0x5d…, 0x6d…, 0x6e…

0x7d… protected and 0xd7… final

I found no ledger statement, artifact, or gate result anywhere that indicates a protected or final seed was read. status.md:11–12 is consistent with the ledger on this point.

However, the declared bank boundaries are too narrow for the constants the sources already use. approaches/ hard-codes protected/final constants far outside the declared banks:

• 0x7d600000, 0x7d630000, 0x7d65c000, 0x7d660000, 0x7d670000, 0x7d690000, 0x7d69c000, 0x7d6a0000, 0x7d6a1000, 0x7d6a4000, 0x7d6a5000, 0x7d6a6000, 0x7d6a7000, 0x7d6b0000, 0x7d6b1000, 0x7d6c1000, 0x7d6d0000, 0x7d6e5000, 0x7d6f0000, 0x7d700000, 0x7d730000
• 0xd7600000, 0xd7630000, 0xd765c000, 0xd7660000, 0xd7670000, 0xd7690000, 0xd769c000, 0xd76a0000, 0xd76a1000, 0xd76a4000– 0xd76a7000, 0xd76c0000, 0xd76c8000, 0xd76d0000, 0xd76e5000, 0xd76f0000, 0xd7738000

A conservative importer must reserve the entire 0x7d…… and 0xd7…… top-byte families, not the two narrow banks named in history.md:181–183.

Importer hazards

1. ASCII domain tags are not seeds. 0x4d435453 = "MCTS", 0x4d504f4c = "MPOL", 0x4d52564c = "MRVL", 0x4d444953 = "MDIS", 0x6e455301, 0x6e4c4301, 0x3d40c0de, 0x3f00c0de are RNG-domain separators, not game seeds. A naive grep-based importer will misclassify them.
2. 0xd7075eed is the fair-D4 policy salt (baselines-v1.json, algorithm.policySeedHex), not a final-cohort game seed. It nevertheless sits inside the 0xd7 final-seed family — a namespace collision worth fixing in a successor manifest.
3. Six ranges have no recorded span and must be reserved generously: Rainbow training (0x3d4…), evo generation batches (0x3d50…), PPO v1 training (0x3f00…), phase-blend tuning (0x5d70…), and the 0x3d7a…/0x3d7b…/0x3d7c…/0x3d7d… oracle label families.
4. Nine complete-game experiments record no seed range at all — see Finding S3.

(c) Findings by severity

CRITICAL

C1 — A frozen protocol embeds a 7,000-point number as the "corrected" D4 reference, making its qualification gate unsatisfiable. artifacts/protocols/optimistic-phase-ntuple/protocol.json:101 requires, at 100 million transitions, that

"corrected D4 reproduces frozen means 176925.25 score and 116.375 moves"

176,925.25 is the 7,000-point Sequence-scored confirmation mean (history.md:970). The corrected-17k mean for the identical cohort and identical trajectories is 400,675.25 (history.md:3337). A corrected-scoring D4 run can never reproduce 176,925.25, so this gate condition is permanently false and the protocol's Stage-B qualification path is dead. The same protocol also demands "search aggregate score and moves are each >= corrected D4 on identical seeds" — which, read together with line 101, is self-contradictory about which "D4" is meant.

AGENTS.md forbids editing frozen protocol artifacts. The correct remedy is a new versioned protocol that names the scoring mode explicitly and cites history.md:3337; the v1 record must be retained with this defect documented.

C2 — Two preregistered gate rejections reverse under corrected scoring. Because the corrected replay proves that the level bonus does not change fair search decisions (section 0), every 7k score-mean gate can be exactly recomputed as score + 2,000·moves − 9,250.

These are not speculative rescorings. They are arithmetic on the ledger's own paired means, using a rescaling the ledger itself demonstrates. The two rejections are therefore scoring-mode artefacts, not scientific results, and docs/strategies.md:157–162 ("more chance samples … did not reliably improve the whole game") and the corresponding experiment-index.md row ("Seven-stratum D4 … score-neutral to worse") overstate the evidence.

For completeness I recomputed the third close call and it does not flip: the 25-move rollout-veto quality extension (history.md:2754–2775) fails its lower-half-score retention gate at 88.38% under 7k and at 89.62% under corrected scoring — narrower, but still below the 90% floor, and its "2 of 3 new pairs" sub-gate is unaffected. Its rejection stands.

I also checked every other paired comparison in the 7k set for a sign change: these two are the only ones. The rest have Δscore and Δmoves of the same sign, so rescoring strengthens rather than reverses them.

C3 — 81 of 119 seven-thousand-point results carry no scoring-mode label, and the ledger's disclaimer selects nothing. history.md:57 scopes the archival warning to "experiments below that identify levelBonus: 7000"; that literal string appears nowhere in the file except in the two disclaimers themselves (history.md:57, 3347). Twenty-three experiment sections — including the entire fair-D4 ablation programme — present 7k scores as if they were corrected-mode results. See section (a) for the enumerated list.

HIGH

H1 — The fitting selection that opened the reveal-reward heldout is entirely produced by one pre-observed game, and reverses on the three genuinely fresh ones. history.md:2166–2184. Seed 0x3def0000 was run first as "a separately persisted diagnostic", frozen byte-for-byte, and then included in the four-game fitting mean that selected the winner. Removing it:

The selected arm is 20% worse on score and 13% worse on moves than stock on the three fitting games that were not already observed; 60.8% of its total fitting score comes from the single pre-observed game. The ledger's own lower quartiles corroborate this (balanced Q25 74,179.50 vs stock 74,049.75 — a 0.2% difference against a 38% mean difference). This exactly predicts the heldout reversal that followed. history.md discloses the pre-observation (2166–2172) but never states the dominance, and the gate was not re-evaluated without the pre-observed seed.

H2 — status.md's headline 64-game D4 reference has no identified cohort, no dispersion statistics, and no censoring statement. The number 308,295.578 / 90.031 appears exactly once in the ledger (history.md:4234), as an internal bootstrap comparator inside the regenerative expert iteration experiment, whose seed lane is 0x3da41000…0x3da7ffff. Which 64 seeds were used is not recorded. docs/methodology.md:69–77 requires mean, median, lower quartile, minimum, move Q25, censor count, clears/reveals per move, chain depth, paired deltas, and confidence bounds for a reported cohort; none of these are recorded for the repository's primary reference number. status.md:8–9, status.md:22, experiment-index.md:41–44, README.md:54, and baselines-v1.json.strengthEvidence.broadMeanScore all propagate it.

H3 — Twelve later "fresh"/"training-only" seed ranges overlap the n-tuple training block. See the 0x3d7… collision table in section (b). Five of them are described in history.md with the word "fresh" (658, 673, 638, 687, 721). The scientific contamination risk is low — the colliding consumers are unrelated fair-expectimax and oracle programs that never saw the n-tuple model — but the ledger's freshness claim is factually wrong, and a seed allocator that imports the ledger at face value will re-issue burned seeds.

H4 — Fourteen sources whose ledger results are 7k-scored have no kLevelBonus == 7'000 build lock. 26 sources do carry the lock; experiment-index.md:35–37 implies the lock protects "several Sequence-scored sources". The unlocked 7k-era sources are: fair-clear-reward-confirmation.cpp, fair-reveal-reward.cpp, oracle-topology-residual-extension.cpp, d4-d2-rollout-veto-quality-extension.cpp, d2-vertical-ladder-probe.cpp, fair-vertical-ladder-depth4.cpp, ntuple-tc.cpp, ntuple-phase-conditioned.cpp, scaled-d4-distill.cpp, transition-reward-horizon.cpp, d4-d2-rollout-veto-exact-compressed.cpp, d4-d2-rollout-veto-cache-free.cpp, d2-rollout-teacher-compression.cpp, d2-long-outcome-ranker.cpp. Rebuilding any of them against the current 17k engine will silently produce numbers that disagree with the ledger. (Three of them assert the 7k pilot totals at replay and would abort loudly, which is the desired behaviour — the remainder would not.)

H5 — The 70,000-point board-clear bonus is unaccounted for everywhere. Neither research/schemas/game-result-v1.schema.json nor the "Standard per-game record" list in docs/benchmarks.md retains a board-clear count or a row-rise count, and no history.md entry reports one. Consequently the score identity cannot be verified per game for any existing or future run, and the large oracle residuals (730–830 points/move, history.md:568, 607, 722, 735) cannot be attributed between deep chains and board clears.

MEDIUM

M1 — history.md self-describes as chronological but is not. Line 3: "the unabridged chronological record". The scoring-mode sequence disproves it: 17k (144–617) → 7k (514–555) → 17k (568–617) → 7k (640–3178) → 17k (3335+). Since the 2026-08-15 scoring error and its correction are dated events, the sections are ordered by topic. Readers using the file to reconstruct what was known when a gate was frozen will be misled.

M2 — Nine complete-game experiments record no seed range at all. The entire "Explicit reservoir and constructive-cycle policies" block (history.md:4389–4456) and "Tail-focused complete-game CEM" (4458–4484) cite SHA-256 values for every source and artifact but no seed range for: vertical-reservoir 128-game tournament (4398), viability-controller 32-game Stage A (4409), constructive-spectrum 32-game Stage A (4421), constructive D4-integration fitting quartet (4425), horizon-scale 32 fitting games (4437), H12 Pareto risk gate (4441), tail-survival-CEM 33,792 candidate-games and 256-game tournament (4465–4466). The 477-root, eight-origin H200 deployment panel that underpins the four most recent conclusions (4257–4604) likewise records the corpus SHA-256 but never the origin game seeds. These cannot be imported into a seed registry.

M3 — d4-phase5-value-veto uses the wrong level bonus as a policy parameter, not merely as a score label. history.md:1998–1999: a challenger must have "a completed-D4 root-Q loss no greater than one canonical 7,000-point level bonus". The same 7,000 band is baked into d4-d2-rollout-veto (2250), its exact-compressed variant (2451), the teacher-compression menu (2581), and the long-outcome veto classifier (2901, 2914). Under corrected scoring the analogous band is 17,000, so these are not just mislabeled results — they are policies whose admission thresholds are calibrated to the wrong constant. Any port must re-derive the band, which makes it a new candidate, not a rescoring.

M4 — status.md:21 presents a rescoring replay as a cohort result. The row "Fair D4 vs D3: 400,675.25/116.375 vs 235,071.25/71 over 8 games … Recorded, small confirmation cohort" does not say that these are the same eight already consumed development seeds (0x3e9c0000…007) replayed under corrected scoring, with byte-identical trajectories. docs/strategies.md:102–103 gets this right ("corrected-score replay on an already-consumed eight-game confirmation cohort"); status.md and README.md:51–54 do not.

M5 — experiment-index.md overstates the rollout-veto pilot. Its row reads "Runtime-paused — ledger-recorded; one promising pilot passed quality gates but missed the runtime limit by a wide margin." history.md:2263 says the opposite: "This is only one pair, not statistical evidence." No quality gate was evaluated on the pilot; the four-game quality gate that was later run failed (history.md:2771–2775). The index also describes the "original 404,000-point pilot" without noting that 404,047 is a single 7,000-point-scored game.

M6 — docs/strategies.md:281–283 quotes the 404,047/250 vs 159,616/105 pilot with no scoring label, inside a document that states at 60–64 that 7,000-point absolute scores "are not evidence for the Hardcore target". Same omission in the "Compact evidence map" row (375).

M7 — Small-n claims. Twenty-three reported comparisons rest on n ≤ 4, and eleven on n = 1. docs/benchmarks.md classifies 1–8 games as PILOT with "no strength claim". Single-game rows: history.md 568, 1465 (×4), 1688–1690 (×5), 1886, 1949–1950, 2003, 2167–2168, 2255–2256, 2763–2766. Four-game rows: 226, 296, 514, 554, 640, 658, 673, 687, 793, 962, 1697, 2176, 2768, 3335, 3707, 4425. The ledger is generally honest about this in prose (e.g. 1823–1825, 1830–1832, 922–923, 2263), but experiment-index.md and status.md frequently carry the resulting numbers forward without the caveat.

M8 — Means dominated by one game, disclosed unevenly. history.md discloses the dominance at 1825 (selective-D5 heldout: "removing the single +201,765-point/+120-move pair made the means negative"), 1831 (screen), 922–923 (fair-only confirmation), and 1162 (CEM screen). It does not disclose it at 2768–2771, where the four-game rollout-veto quality mean gain of +67,652 falls to +8,725 once the frozen pilot game is removed — a 7.8× dependence on one game — while the text says only "all four leave-one-out subsets were positive" (technically true, materially misleading). And it does not disclose it at 2176–2178 (Finding H1).

M9 — The 0x4d… family has two contradictory declared roles. history.md:179–180 declares 0x4d70… and 0x5d70… as "historical development/tuning ranges that have already been observed"; history.md:626 and 4099 instead treat the whole 0x4d byte as protected ("every protected 0x4d, 0x7d, and 0xd7 cohort remain unopened"). Both cannot be true: 0x4d700000…3f (103, 137, 156) and 0x4d400000…3f (3477, 3480) were read. status.md's protected/final assertion is unaffected — those are 0x7d/0xd7 — but the seed registry must not inherit the "0x4d is protected" wording.

LOW

L1 — history.md:462–469 is internally contradictory about the denoised-value screen. Line 462 calls it "that legacy-scoring run"; line 467–468 says "its reported absolute point totals use the verified 17,000-point Hardcore bonus". No point totals are reported for that screen or confirmation at all — only move means (78.75/81.25 and 79.5/88.125). The scoring claim is therefore unverifiable from the ledger, and experiment-index.md's "a small confirmation improved means" inherits the ambiguity.

L2 — history.md:891–903 makes a scoring claim about a pilot whose scores were never recorded. "The historical 155/160-move pilot … Its 17,000-point bonus is now verified as correct." Move counts, clears, and reveals are given; no score is. The section also correctly notes n = 2 non-independent training seeds.

L3 — 0xd7075eed (fair-D4 policy salt) sits inside the 0xd7 final-cohort family. No leakage: it is a policy-derived salt, never a game seed. But it will be misclassified by any automated seed importer and should be moved out of the reserved byte in a successor manifest.

L4 — status.md:18 ("122 local tests") was not re-verified by this audit; verifying it requires executing the test suite, which is outside a read-only claim-arithmetic audit.

L5 — Numbers that do check out. For the record, I recomputed and confirmed: the quality-extension four-game means, lower halves, and retention percentages (history.md:2763–2773, all exact); the corrected-replay paired gains of +81,423 / +22 and +165,604 / +45.375 (3336, 3338); and the D2/D4 12-game diagnostic move mean of 77.17 = 926 roots / 12 games (2092, 2104). No arithmetic error was found in any of them.

(d) Claims that should be downgraded, with proposed wording

The coordinator owns these edits; none has been applied.

(e) Verdict

Assessed as an audit of the historical evidence ledger, not of any single experiment.

• Run validity of the ledger as a record: partial. The arithmetic is sound — 205 of 205 reported results are internally consistent, with zero transcription errors and zero mode-ambiguous rows — but the record is incomplete in three ways that materially affect interpretation: missing scoring-mode labels on 81 of 119 archival results (C3), missing cohort and dispersion statistics for the headline reference number (H2), and missing seed ranges for nine complete-game experiments plus the 477-root deployment panel (M2).

• Scientific outcome of the audit: fail on claim integrity, pass on computational integrity. Two preregistered rejections (fair-depth4-s7, fair-cem-optimizer) do not survive correction to the mandated 17,000-point bonus and must be reopened as unresolved (C2). One frozen protocol contains an unsatisfiable gate (C1). One fitting selection was driven by a pre-observed game and reverses without it (H1).

• Highest justified evidence tier for fair D4 as the reference policy: development — and, strictly, reusable/burned development. The 8-game corrected figure is a rescoring replay of already-consumed development seeds; the 64-game corrected figure has no identified cohort and no dispersion statistics. Neither reaches STANDARD under docs/benchmarks.md, and baselines-v1.json is correct not to assign one.

• Protected and final cohorts: I found no evidence anywhere in the ledger, artifacts, or sources that a 0x7d…… or 0xd7…… game seed was ever read. status.md:11–12 is supported. The declared bank boundaries are, however, narrower than the constants the sources already reserve, and must be widened before a registry is written.

• The million-point claim: unchanged and unsupported. The only recorded above-million means are the privileged future-aware oracle (history.md:735, 12 games, censored at 500 moves, 7,000-point scored) and its 500-move single-seed run (history.md:568, 17k). No public-information policy has produced a mean above 400,675.25 in any scoring mode on any cohort. The 1,246,684-point single game (status.md:23, README.md:58) is unverifiable and unclassifiable and should be downgraded per D3.

Audit inputs

docs/research/history.md (all 4,607 lines) · docs/research/status.md · docs/research/experiment-index.md · docs/strategies.md · docs/methodology.md · docs/benchmarks.md · README.md · src/core/native/engine.hpp · src/core/native/ntuple.hpp · research/schemas/game-result-v1.schema.json · research/benchmarks/baselines-v1.json · research/seeds/README.md · artifacts/protocols/optimistic-phase-ntuple/protocol.json · seed and kLevelBonus constants across approaches/.

No cohort was opened, no gameplay was run, and no file other than this one was created or modified.