Fifty million moves, then a two-rise look-ahead

The family's largest training run: a pattern evaluator trained on 50 million moves, then searched two row-rises deep — and the search made it worse than just trusting the network.

The intuition

Two ideas, stacked.

The first is about what to learn. Nearly everything else in this family regressed the value of the one column the policy happened to play, then had to guess about the six it did not. This experiment instead learns the value of a position directly, before the next disc is dealt — a chance-state value — so the same learned number serves every column that leads there. It also starts pooled: all five positions in the rise cycle share one set of tables, and only after 20 million moves are the weights copied into five separate phase-specific heads. Pooling first means the rare phases get data from the common ones before they are allowed to specialise.

The second is about how far to look. Drop7's real deadline is the row rise every five drops. A search that stops in the middle of a cycle is judging a board that is about to be shoved upward. So this policy deepens iteratively across one rise boundary and then two, keeping every legal column at the root and admitting the best two deeper.

To make two-rise search affordable, it does not average over the whole chance tree. For each bundle of sampled outcomes it follows the single sampled outcome whose value is closest to the bundle's average, and continues down that line. That is a bounded, deterministic representative-outcome rollout, not an exact fair expectimax — a deliberate approximation, and the one that turned out to matter.

How it works, step by step

1. Train pooled. On-policy temporal-difference learning over 184 active four-cell n-tuple windows, reflection-canonicalised, with an undiscounted three-step forward view.
2. Split by rise phase at 20 million moves. Copy the pooled weights into five phase-specific heads and keep training.
3. Stabilise at the end. Adaptive per-weight rates are enabled only for the last ten million moves of the budget.
4. Play. For each legal column, take all seven coordinate-stratified gray-disc reveal outcomes; deepen across one rise boundary, then two; follow the representative outcome down each line; finish both boundaries inside 100,000 simulator calls per decision. A decision that cannot finish is a fatal error, not a fallback.

What happened, in plain English

Training completed exactly as planned: 50 million moves across 1,057,844 complete games, with no intermediate checkpoint chosen and nothing selected on results.

Then the mandatory gate ran, and it rejected the whole family decisively. Just playing the network's own first choice scored 181,733 points and 56.4 moves per game. Adding the two-rise search — the entire point of the design — dropped it to 113,644 points and 37.4 moves. It was worse in both halves of the cohort, on both score and moves. The gate required at least 300,000 points and 90 moves; the search was nowhere near either.

The failure is not a resource excuse. Every decision completed inside its work budget, no move was illegal, and no game was cut short. The search genuinely believed in worse moves.

The technical record

Status in the experiment index: rejected, ledger-recorded — "after 50 million transitions the representative-outcome search was worse than direct n-tuple play."

From the ledger, on the burned 64-game cohort 0x3d200000...0x3d20003f:

Policy	Mean score	Mean moves
Direct n-tuple play	181,733.422	56.359
Two-rise-boundary search	113,643.969	37.375

Ordered halves: direct 180,667 / 182,800 points and 56.125 / 56.594 moves; search 118,367 / 108,921 points and 38.719 / 36.031 moves. The final training chunk averaged 176,247 points and 54.811 moves.

Training: exactly 50,000,000 transitions over 1,057,844 completed games in 1,504.404 seconds, in resumable chunks of at most 5 million, with no intermediate evaluation or checkpoint selection. The phase model with adaptive accumulators uses 65.4 MB of parameter storage. The fit lane was the previously unused seed family 0x6d000000...0x6dffffff. The 256-game development cohort 0x6e000000...0x6e0000ff and every protected and final cohort remain unopened; no qualification artifact was written and training stopped before the 100-million-move continuation.

Provenance is unusually tight for this family: two independent semantic reviews before the source was frozen, strict Clang 21 and GCC 14 optimized executable and library builds, sanitizer runs, and an immutable preregistration at artifacts/protocols/optimistic-phase-ntuple/protocol.json.

A recorded defect in that frozen protocol. docs/exploratory/audit-03-claim-arithmetic.md records as its most severe finding (C1) that the protocol's 100-million-move qualification clause requires a corrected-scoring depth-4 search to "reproduce frozen means 176925.25 score and 116.375 moves". That score is the historical 7,000-point mean of an eight-game confirmation; the corrected-scoring replay of the identical trajectories is 400,675.25. A corrected-score run can never reproduce it, so that clause of the gate is permanently unsatisfiable. The audit's own remedy is a new versioned protocol, because frozen artifacts are not edited. This did not affect the Stage-A rejection above, which used absolute floors rather than that clause.

Scoring mode. The ledger section does not name the level bonus. The same audit classifies all of this experiment's rows as consistent with the corrected 17,000-point award.

Source: optimistic-phase-ntuple.cpp.

What this taught us, and what is still open

Search over a weak evaluator can be worse than no search. This is the sharpest demonstration of it in the repository: identical weights, identical cohort, and adding two rise-cycles of look-ahead cost roughly a third of the score. A deeper search compounds the leaf evaluator's errors instead of averaging them away, and it also chases the approximation used to keep the tree affordable — here, following one representative outcome rather than averaging over chance, which is precisely the "unfair" chance handling the repository has repeatedly measured as harmful.

The training scale was not the problem. 50 million moves of on-policy learning produced a policy roughly level with much cheaper ones elsewhere in this family. See is more computation the answer?.

What it did not rule out. It rejects this configuration: this target, this pooled-then-split schedule, and this representative-outcome rollout. It says nothing about a phase-conditioned n-tuple used as the leaf of an honest fair search, which was never run.