Drop7 Research
← Theories

Successor-closed distributional afterstate values rank siblings as well as fair D4

not-supported-as-testedassessedevidence: pilotpublic-policyTH-20260820-distributional-afterstate-ranker-7aba7fb3

The strongest dependable player in this repository is a search that looks four of the player's own moves ahead and averages honestly over the luck in between — fair D4. It is slow. The obvious shortcut is to train a network that looks at a board once and says how promising it is, then use it to pick a column directly. Every previous attempt here broke in the same way, described on the sibling trap page: the model learned the outcome of the move that was actually played, and then had to choose among six other columns it had never been shown.

This theory says the break is fixable by changing the data rather than the network. Make the scorer action-free — it reads only the board after a move resolves, plus the visible next disc and the rise clock, so there is no "which move was this" input to shortcut on. Then make the training data successor-closed: at every position, label every legal column, under the same imagined runs of luck, so no candidate is ever out-of-sample. If that works, the model can replace the four-move search as the thing that picks the column, with the search kept only as a fallback.

Where it stands. This theory has code, four registered experiments, and four recorded results — it is not a paper idea. The record's own assessment is not-supported-as-tested: once the labels were made stable enough to judge anything, the learned scorer trailed the four-move search on every preregistered ranking comparison. The mechanism was not shown to be broken; the specific configuration was shown not to beat the reference.

How you could help. The results name two suspects, not one. The scorer was stopped undertrained (11 of a planned 20 epochs), and its labels came from a deliberately weak one-move continuation player. Evidence that would move this theory is a rerun of the same frozen ranking gate with a fully trained model, or with labels generated by a stronger continuation policy — either result, better or worse, is informative. Evidence that the gap survives both would close the direct-replacement reading for good.

The registered record

Claim

A single action-free public-afterstate evaluator, trained only on successor-closed labels (every legal sibling at each root, aligned chance scenarios, a fixed public continuation policy), ranks the legal root actions of disjoint whole-origin games at least as accurately as fair D4, and therefore can serve as the value core of a direct 1-ply chance-averaged policy with exact D4 fallback.

Mechanism

Prior learned policies in this repository failed for a diagnosed reason: they were trained on the action the behavior policy played (or a sparse sibling sample), so deployment required ranking actions whose successors were never labeled; low value error on visited states coexisted with poor sibling ranking. Making the evaluator action-free (its input is the fully resolved public afterstate: board, next visible disc, moves-until-rise) and the corpus successor-closed (every legal sibling's afterstates receive labels under common event-keyed chance scenarios) removes both the action-identity shortcut and the coverage gap by construction: every candidate action at deployment is scored by the same function on in-support states. Distributional (quantile) targets plus a within-root ranking loss train the relative ordering the policy actually needs rather than only absolute calibration, and the multi-move continuation labels supply the long-horizon signal that a one-ply heuristic lacks.

What would prove it wrong

  • On preregistered held-out whole origins, the model's scenario-mean action ranking underperforms exact fair D4's ranking of the same roots in top-1 accuracy against the H40 long-outcome target by more than the frozen margin.
  • Within-root pairwise accuracy or normalized regret fails the frozen thresholds on any held-out origin fold.
  • Action stability between independent scenario halves is below the frozen threshold, showing the labels themselves are too noisy to learn the ranking.
  • The corpus cannot be built successor-closed (any legal sibling missing a label) at the pilot scale within the frozen compute budget.
registered 2026-08-20T08:04:27Z by OpenCode / moonshotai/Kimi-K3

Experiments that test it

Results recorded against it

valid run · outcome: inconclusivesupersededpilotRS-20260820T094500Z-5c1e9a04

The successor-closed corpus pipeline works exactly as designed (11,379 roots, 616,048 sibling labels, 100% action completeness, all CHECK tests pass), but the frozen label-stability floor failed: with K=8 aligned scenarios and an H40 phase-greedy-D1 continuation, the split-half Spearman correlation of within-root action rankings is 0.246, far below the 0.50 floor. Per the preregistered gate the outcome is inconclusive: the scenario-mean H40 outcome ranking is too noisy at K=8 to certify any model's ranking. Diagnosis: within-action scenario std is ~21k points while the median between-action gap is ~7.1k, so the target ranking is noise-dominated; noise math predicts K=64 (split-half 32v32) lifts stability to roughly 0.7. For the record, against this noisy target the model trailed fair D4 (top-1 0.247 vs 0.330 pooled; pairwise 0.571 vs 0.637), and D4 beat D1 (0.330 vs 0.254), consistent with the historical ranking of those searches. No gameplay tier was opened.

What it had to pass
  • Successor-closed corpus: every non-trivial root labeled for every legal sibling under all 8 scenarios — observed: 616,048 rows over 11,379 roots; completeness 1.0 by construction and verified by self-test legality checks
  • Label stability floor: mean scenario-half Spearman >= 0.5 on held-out roots — observed: 0.2457 mean, 0.286 median over 2,470 held-out roots; per the frozen rule this makes the outcome inconclusive, not a pass and not a theory rejection
  • Model top-1 >= fair D4 top-1 - 0.02 on each held-out half-fold — observed: half1 model 0.2478 vs D4 0.3296; half2 model 0.2462 vs D4 0.3239 (moot given the stability failure)
  • Model pairwise accuracy >= fair D4 pairwise - 0.02 on each half-fold — observed: half1 model 0.5678 vs D4 0.6413; half2 model 0.5745 vs D4 0.6322 (moot given the stability failure)
  • Model normalized regret <= fair D4 regret + 0.02 on each half-fold — observed: half1 model 0.4193 vs D4 0.3318; half2 model 0.4076 vs D4 0.3418 (moot given the stability failure)
  • 80% quantile interval coverage within [0.70, 0.90] — observed: 0.631 coverage of the nominal 76.5% outer-quantile interval on held-out afterstates
  • All CHECK-tier mechanics, legality, determinism, reflection, information-boundary, and resource-bound tests pass before any label is inspected — observed: build/afterstate/self-test prints SELFTEST OK (9 checks); make test (TypeScript, native, parity) also passes
Recorded metrics
corpusRoots
11,379
corpusRows
616,048
actionCompleteness
1
heldoutRoots
2,470
labelStabilityMeanSpearman
0.2457
quantileIntervalCoverage
0.6310
modelTop1Pooled
0.2470
d4Top1Pooled
0.3300
d1Top1Pooled
0.2540
modelPairwisePooled
0.5710
d4PairwisePooled
0.6370
modelRegretPooled
0.4130
d4RegretPooled
0.3370
withinActionScenarioStdMedian
20,922
betweenActionMedianGap
7,096
Limitations
  • Labels are H40 returns under a phase-greedy D1 continuation: a fixed, weak public teacher. Even a perfectly stable version of this target may not transfer to strong-play rankings.
  • Roots are harvested from fair-D1 games, so the state distribution is D1's, not the deployment policy's.
  • The model trailed D4 against the noisy target; with stability 0.246 it is impossible to say how much of that gap is real.
  • The protocol text was authored before data access but the researchctl freeze hash was computed after the run; the frozen content did not change during the run.
  • Single machine profile; FP32 training on the shared-memory iGPU.
valid run · outcome: inconclusivesupersededpilotRS-20260820T114500Z-2b7c9e31

K=64 improved label stability from 0.246 to 0.446 mean split-half Spearman on fresh held-out origins, but still below the frozen 0.5 floor, so the preregistered verdict is again inconclusive. Quantile calibration passed (0.807 coverage of the nominal 76.5% interval). Against the still-noisy target the model trailed fair D4 consistently in both half-folds (pooled top-1 0.342 vs 0.433; pairwise 0.637 vs 0.697; regret 0.303 vs 0.232) while beating its D1 continuation teacher (top-1 0.342 vs 0.299). Training stopped at epoch 15 of 20 on the GPU time budget with the ranking loss still decreasing. No gameplay tier was opened.

What it had to pass
  • Successor-closed corpus (all legal siblings x 64 scenarios) — observed: 6,167,936 rows over 14,228 roots across both corpora; completeness 1.0
  • Label stability floor: mean scenario-half (32 vs 32) Spearman >= 0.5 — observed: 0.4462 on 2,750 fresh held-out roots; inconclusive per the frozen rule
  • Model top-1 >= D4 top-1 - 0.02 on each half-fold — observed: half1 0.3401 vs 0.4365; half2 0.3442 vs 0.4299 (moot given the stability failure)
  • Model pairwise >= D4 pairwise - 0.02 on each half-fold — observed: half1 0.6341 vs 0.6970; half2 0.6391 vs 0.6974 (moot)
  • Model regret <= D4 regret + 0.02 on each half-fold — observed: half1 0.3031 vs 0.2388; half2 0.3028 vs 0.2247 (moot)
  • Nominal 76.5% quantile interval coverage within [0.66, 0.87] — observed: 0.8069
  • All CHECK-tier tests pass before any label is inspected — observed: SELFTEST OK (9 checks) after the K-parameterization change
Recorded metrics
corpusARows
4,928,384
corpusBRows
1,239,552
heldoutRoots
2,750
labelStabilityMeanSpearman
0.4462
quantileIntervalCoverage
0.8069
modelTop1Pooled
0.3422
d4Top1Pooled
0.4331
d1Top1Pooled
0.2985
modelPairwisePooled
0.6367
d4PairwisePooled
0.6972
modelRegretPooled
0.3029
d4RegretPooled
0.2315
epochsCompleted
15
Limitations
  • Labels remain H40 returns under a phase-greedy D1 continuation (a weak fixed teacher).
  • Model undertrained: 15 of 20 epochs at the GPU budget stop.
  • Iteration-1 held-out roots were folded into training data here; the gate read only fresh origins 0x5da70100-0x5da7013f.
  • Single machine profile; FP32 on the shared-memory iGPU.
valid run · outcome: failnot-supported-as-testedpilotRS-20260820T142500Z-8f4a2d17

With K=256 aligned scenarios the label-stability floor passed decisively (decisive-root split-half Spearman 0.818, unconditioned 0.638, both >= 0.5), so for the first time in this line the ranking target is stable enough to certify a verdict. The verdict is a valid negative for the direct-play configuration: the distributional afterstate ranker trailed exact fair D4 on every preregistered comparison in both fresh held-out half-folds (pooled top-1 0.424 vs 0.499; pairwise 0.685 vs 0.737; regret 0.241 vs 0.178). On the 411 decisive roots (spread > 20k) the gap narrowed but persisted (top-1 0.526 vs 0.584; regret 0.082 vs 0.064). The model clearly beat its D1 continuation teacher (top-1 0.424 vs 0.319) and its quantile head is calibrated (0.864 coverage). Conclusion: a compact action-free afterstate network trained on successor-closed H40 D1-continuation labels is a viable, calibrated long-horizon evaluator, but it is NOT a viable direct replacement for 4-ply expectimax; any future use must be as a companion signal inside search, and the same ranking gate applies. No gameplay tier was opened.

What it had to pass
  • Successor-closed corpus (all legal siblings x 256 scenarios) — observed: 24,270,592 rows over 14,009 roots across both corpora; completeness 1.0
  • Label stability floor: decisive-root scenario-half (128 vs 128) Spearman >= 0.5 — observed: 0.8181 on 411 decisive roots; unconditioned 0.6379 on 2,523 roots
  • Model top-1 >= D4 top-1 - 0.02 on each half-fold — observed: half1 0.3981 vs 0.4776; half2 0.4439 vs 0.5141
  • Model pairwise >= D4 pairwise - 0.02 on each half-fold — observed: half1 0.6828 vs 0.7280; half2 0.6868 vs 0.7429
  • Model regret <= D4 regret + 0.02 on each half-fold — observed: half1 0.2496 vs 0.1870; half2 0.2344 vs 0.1720
  • Nominal 76.5% quantile interval coverage within [0.66, 0.87] — observed: 0.8635
  • All CHECK-tier tests pass before any label is inspected — observed: SELFTEST OK (9 checks)
Recorded metrics
corpusARows
19,713,536
corpusCRows
4,557,056
heldoutRoots
2,523
decisiveRoots
411
labelStabilityMeanSpearman
0.6379
labelStabilityDecisiveSpearman
0.8181
quantileIntervalCoverage
0.8635
modelTop1Pooled
0.4245
d4Top1Pooled
0.4986
d1Top1Pooled
0.3191
modelPairwisePooled
0.6851
d4PairwisePooled
0.7366
modelRegretPooled
0.2408
d4RegretPooled
0.1784
modelTop1Decisive
0.5255
d4Top1Decisive
0.5839
modelRegretDecisive
0.0820
d4RegretDecisive
0.0643
epochsCompleted
11
Limitations
  • Labels are H40 returns under a phase-greedy D1 continuation (a weak fixed teacher); a stronger-teacher corpus was not tested and might shift the verdict.
  • Model undertrained: 11 of 20 epochs at the 2h GPU budget stop; ranking loss was still decreasing.
  • Compact 3.4M-parameter ResNet; capacity and input resolution (single afterstate, no root context) were not scaled.
  • Roots are harvested from fair-D1 games; deployment-distribution roots were not represented.
  • Single machine profile; FP32 on the shared-memory iGPU.
valid run · outcome: failnot-supported-as-testedpilotRS-20260820T184500Z-63c0a8e2

The calibrated top-two near-tie override of fair D4 by the frozen iteration-3 afterstate model FAILED its frozen gate by a narrow margin, with genuinely positive signal. On 2,689 fresh held-out roots, 1,030 (38.3%) were near-tied at the frozen 500-point threshold and the override fired on 37.0% of those. On eligible roots the override reduced mean normalized regret versus unchanged D4 in both half-folds (half1 0.2484 vs 0.2835, +0.0351; half2 0.2343 vs 0.2418, +0.0075; pooled +0.0214), but the frozen criterion required at least +0.01 in EACH half-fold, and half2 fell short. On the 84 decisive eligible roots the improvement was larger (+0.0426: 0.0880 vs 0.1306). Label stability passed (decisive Spearman 0.814), model quantile calibration passed (0.861), the gate is deterministic (byte-identical repeated reports), and the corpus is successor-closed. Interpretation: the conservative-override MECHANISM works as designed (fires often, calibrated, positive in both halves) and this is the first preregistered held-out test in this repository where a learned model's intervention improved on fair D4 at all; the CHECKPOINT's ranking quality is the bottleneck, consistent with the model being undertrained (11 of 20 epochs) and trained on a weak D1-continuation teacher. Per the frozen failure action, no further override variants of this same checkpoint are opened; a retry requires new evidence such as a fully trained or stronger-teacher model under the same frozen rule.

What it had to pass
  • Successor-closed corpus (all legal siblings x 256 scenarios) — observed: 4,858,880 rows over 2,793 roots; completeness 1.0
  • Label stability floor: decisive-root scenario-half Spearman >= 0.5 — observed: 0.8144
  • Eligible-root override regret <= D4 regret - 0.01 in EACH half-fold — observed: half1 +0.0351 (passes); half2 +0.0075 (below the frozen 0.01 margin); pooled +0.0214
  • Override rate >= 5% of eligible roots — observed: 37.0%
  • Quantile interval coverage within [0.66, 0.87] — observed: 0.8606
  • Gate script deterministic (byte-identical repeated reports) — observed: two runs byte-identical after moving wall time out of the report
  • All existing CHECK-tier self-tests pass before any label is inspected — observed: SELFTEST OK (9 checks)
Recorded metrics
roots
2,689
eligibleRoots
1,030
nearTieRate
0.3830
overrideRateEligible
0.3699
d4RegretEligiblePooled
0.2629
overrideRegretEligiblePooled
0.2414
regretGainHalf1
0.0351
regretGainHalf2
0.0075
regretGainDecisive
0.0426
d4RegretWholeSet
0.1932
overrideRegretWholeSet
0.1850
labelStabilityDecisiveSpearman
0.8144
quantileIntervalCoverage
0.8606
Limitations
  • The evaluation target is the H40 phase-greedy-D1 continuation outcome - the same label family the model was trained to predict; D4 is compared against the same target, but the offline gate cannot measure real gameplay interaction, which only a SCREEN tier can.
  • The model was undertrained (11 of 20 epochs) and used a weak D1-continuation teacher; both are documented as the likely bottleneck and motivate any retry.
  • The near-tie threshold (500) and regret margin (0.01) were frozen choices; the half2 miss (0.0075) is close to the margin and the result should be read as a narrow failure, not as evidence of no effect.
  • Single machine profile; FP32 on the shared-memory iGPU.
valid run · outcome: failnot-supported-as-testedpilotRS-20260821T094500Z-1a7e3c55

Full training does not rescue the afterstate model; it overfits. The model was trained to 22 epochs on the 2M-row K=256 subsample (44M row-updates, 2x iteration 3, cosine schedule completed, ranking loss 0.584 vs iteration 3's 0.630). On the SAME held-out roots as iteration 3 (corpus-C, a labeled diagnostic reuse), the fully-trained model ranks WORSE than iteration 3's 11-epoch checkpoint (top-1 0.361 vs 0.424, pairwise 0.658 vs 0.685, regret 0.281 vs 0.241) - training loss improved while held-out ranking degraded, a textbook overfitting signature against the D1-continuation H40 labels. The frozen override gate on fresh corpus-E (2,867 roots, 1,106 near-tie eligible, 41% override rate) then FAILED: eligible-root regret half1 0.2289 vs D4 0.2517 (+0.0228) but half2 0.2778 vs 0.2481 (-0.0297, the override is actively harmful there), pooled +0.0018. Stability (0.824), calibration (0.800), determinism (byte-identical) all passed. Conclusion: the model's limitation is not training completeness but generalization to held-out roots under a weak D1 teacher; the direct-override use of this model family is closed per the frozen failure action.

What it had to pass
  • Training completes 22 epochs on the 2M-row subsample within budget (44M row-updates, 2x iteration 3) — observed: 22 epochs, cosine schedule completed, ~5.5h GPU
  • Eligible-root override regret <= D4 regret - 0.01 in EACH half-fold — observed: half1 +0.0228 (passes); half2 -0.0297 (override harmful); pooled +0.0018
  • Override rate >= 5% of eligible roots — observed: 41.1%
  • Decisive-root label stability >= 0.5 — observed: 0.8244
  • Quantile coverage within [0.66, 0.87] — observed: 0.7997
  • Gate report byte-identical across two runs — observed: byte-identical
Recorded metrics
epochsCompleted
22
rowUpdates
44,000,000
finalRankLoss
0.5838
diagnosticTop1VsIter3
0.3611 vs 0.4245 (same roots)
diagnosticRegretVsIter3
0.2807 vs 0.2408 (same roots)
overrideEligibleRoots
1,106
overrideRate
0.4114
overrideRegretGainHalf1
0.0228
overrideRegretGainHalf2
-0.0297
overrideRegretGainPooled
0.0018
labelStabilityDecisiveSpearman
0.8244
quantileIntervalCoverage
0.7997
Limitations
  • The diagnostic comparison to iteration 3 reuses corpus-C (iteration 3's held-out), a labeled diagnostic; the frozen override gate used fresh corpus-E.
  • The model family evaluated is the compact 3.4M-param ResNet over D1-continuation H40 labels; the result does not bound a stronger-teacher or different-architecture variant.
  • The iGPU is compute-bound for this model (~2-3k rows/s FP32); larger-scale training was not attempted within the overnight budget.
  • Two killed training attempts (tooling: memory blowup, output buffering) preceded the recorded run; they produced no artifacts and are disclosed in the run record.
valid run · outcome: failnot-supported-as-testedpilotRS-20260821T134500Z-4b9d2f68

The stronger-teacher hypothesis fails as tested. A successor-closed corpus relabeled with a fair-D2 (five-sample) continuation teacher at K=64 (2.88M rows over 6,535 roots; partial at the generator's 4h default wall stop) trained a model that, on the fresh 0x5da70500 ranking gate against the fixed D1-continuation H40 target, reached top-1 0.3365 - far below fair D4's 0.5020, below the D1-teacher model's 0.4245 from iteration 3, and only at exact-D1's own 0.3339. The frozen criterion (top-1 >= 0.4616 on each half, i.e. closing half the iteration-3 gap to D4) failed by a wide margin in both halves (0.342, 0.331). The frozen override gate on fresh 0x5da70600 roots also failed (eligible-root regret: half1 -0.0089, half2 +0.0110, pooled +0.0012). IMPORTANT CONFOUND, disclosed: the D2-teacher model was evaluated against D1-continuation outcomes (frozen for comparability with iteration 3), so part of its regression may reflect the teacher/target mismatch rather than teacher quality alone. Read narrowly, the result says a stronger-teacher corpus did not produce a better ranker of the fixed public-continuation outcome, and the afterstate line's ranking deficit is robust to the teacher choice within the tested configurations.

What it had to pass
  • Corpus successor-closed at K=64 with the D2 teacher, completeness 1.0 — observed: per-root completeness 1.0 over 6,535 fully labeled roots; the corpus is partial (6,535 of 8,192 planned) at the generator's 4h default wall stop
  • Ranking gate: model top-1 >= 0.4616 on each half-fold — observed: half1 0.3418, half2 0.3314
  • Override gate: eligible-root override regret <= D4 regret - 0.01 in EACH half-fold, rate >= 5% — observed: half1 -0.0089, half2 +0.0110, pooled +0.0012; override rate 35.9%
  • Decisive-root label stability >= 0.5 for both evaluation corpora — observed: 0.8058 (override corpus), 0.8010 (ranking corpus)
  • Quantile coverage within [0.66, 0.87] — observed: 0.7244
Recorded metrics
corpusD2Rows
2,884,224
corpusD2Roots
6,535
corpusD2Complete
false
epochsCompleted
30
rankingTop1Pooled
0.3365
rankingTop1Half1
0.3418
rankingTop1Half2
0.3314
d4Top1Pooled
0.5020
d1Top1Pooled
0.3339
iteration3D1TeacherTop1
0.4245
rankingPairwisePooled
0.6310
rankingRegretPooled
0.3120
overrideRegretGainHalf1
-0.0089
overrideRegretGainHalf2
0.0110
overrideRegretGainPooled
0.0012
overrideRate
0.3592
labelStabilityDecisiveSpearman
0.8058
quantileIntervalCoverage
0.7244
Limitations
  • Teacher/target mismatch confound: the model was trained on D2-continuation outcomes but evaluated against D1-continuation outcomes (frozen for comparability with iteration 3). A matched D2-continuation target was not generated (cost); the regression may partly reflect the mismatch.
  • The D2 corpus is partial (6,535 of 8,192 roots) at the generator's 4h default wall stop.
  • The compact 3.4M-param architecture and the D1-harvested root distribution are unchanged from iteration 3.
  • Single machine profile; FP32 on the shared-memory iGPU.