Depth-4 distillation

Before trying to learn something better than the reference search, check the easier thing first: can a small, fast model learn to imitate the search we already have?

The learned part failed. The comparison it was measured against — a plain depth-2 search — turned out to be excellent, and that is the result this experiment is actually remembered for.

Why imitate a search you already have?

Depth-4 search is expensive: roughly 1.35 million units of work per move in this repository's accounting. Almost every ambitious idea in the project needs to evaluate many candidate lines, and it cannot afford depth 4 inside a loop. So it matters a great deal whether depth 4's judgement can be compressed into something cheap.

It is also a control experiment. If a model cannot reproduce a deterministic function of the public board — which is exactly what depth 4 is — then the repeated failures of models trying to learn something harder cannot be blamed on the horizon or on the noise in long outcomes. Imitation is the easy case.

How it works, step by step

1. Take the preserved file of depth-4 root valuations: for each position, the full vector of values that depth 4 assigned to every legal column.
2. Design the architecture using only the already-consumed split, so no design decision is made against fresh data.
3. Freeze everything before a new seed is opened: the anchor, the feature layout, 100 training epochs, batch size, learning rate, regularisation, loss weights, and every acceptance threshold.
4. Anchor the model on exact depth-2 search and learn only a residual — a small correction on top of it — expressed relative to the action, and averaged over the board and its mirror image so the model cannot treat a reflected position as a different one.
5. Collect fresh games, split by whole game, and score both the anchor and the anchor-plus-residual on how often they pick depth 4's own top column, how often they order pairs of columns correctly, and how much value they leave on the table.

What happened, in plain English

The cheap exact search was the surprise. On held-out games, plain depth-2 search picked depth-4's own best column 60.5% of the time and ordered pairs of columns correctly 76.7% of the time — and it costs about 693 times less work per decision than depth 4 does. That is a strong, ledger-recorded ranking diagnostic and it is why depth 2 became the standard cheap continuation throughout the rest of this family.

The learned residual added nothing. It reached 59.4% top-1 and 76.1% pairwise — slightly worse than the anchor on all three accuracy measures — and improved the leftover-value measure by 1.29% against the 5% the protocol had demanded in advance. It also failed the required improvements inside each half of the held-out games. Its 13,216-byte checkpoint exists and is not worth loading.

An earlier, unanchored attempt to clone depth 4 had already failed much more severely — training accuracy 0.765 collapsing to 0.247 on held-out positions. Anchoring on exact search fixed that generalisation failure completely. What it could not do was add anything on top.

The technical record

Architecture design used only the already-consumed 1,508 / 465 root-value artifact. On that fitting-only split: exact depth 1 reproduced depth 4 at 48.60% top-1 and 69.68% pairwise; full-width depth 2 reached 56.77% / 76.68% with 0.14103 normalized regret; the 1,647-weight action-relative sparse residual reached 55.70% / 76.11% / 0.13461.

Frozen run: 1,885 roots from 24 complete fitting games 0x3df20000...017; 926 roots from 12 complete held-out games 0x3df30000...00b. No game censored at the 250-move cap.

Held-out cohort	Top-1	Top-2	Pairwise	Normalized regret
Exact depth 2, all 12 games / 926 roots	60.475%	76.782%	76.701%	0.13192
Exact depth 2, first six games	64.444%	78.788%	76.936%	0.11911
Exact depth 2, second six games	55.916%	74.478%	76.434%	0.14662
Depth 2 + learned residual, all 12	59.395%	76.458%	76.119%	0.13021

Acceptance required at least 55% top-1, 70% top-2 and 72% pairwise overall, the same in each six-game half, regret at most 0.18, and improvements over the anchor of one point top-1, half a point top-2 and pairwise, and 5% regret in the full cohort and both halves. The residual met the absolute thresholds and failed the improvement thresholds.

Cost: exact depth 2 used 1,949.21 logical work units, 983.49 nodes and 5.33 cache hits per root, maximum 2,485 work units, running at 435.39 roots per second; the residual needed both depth 1 and depth 2, 2,016.88 work units per root, at 421.11 roots per second. Against depth 4's roughly 1.35 million work units per move, exact depth 2 is about 693 times cheaper in this accounting.

Records: experiment index "D4 and long-outcome research"; ledger section "Scaled fair-D4 distillation and exact-D2 rollout primitive". Source: scaled-d4-distill.cpp.

What this taught us, and what is still open

• Anchor on something exact. The difference between the failed clone (0.247 held-out top-1) and this one (59.4%) is almost entirely that this model only had to learn a correction to a search that already works. Later experiments reuse the pattern, including one that starts a network as the exact zero residual over depth 4 so the untrained model is the reference policy.
• Learning did not beat searching, even at imitation. With the easy case set up as favourably as possible — a deterministic target, full valuation vectors, a good anchor — the learned part still could not improve on cheap exact search. That is a meaningful negative for the whole "compress the search" direction.
• Depth 2 is the reusable asset. It became the continuation inside the rollout veto and the label generator for the long-outcome corpora on the strength of this ranking audit.
• Still open: the ranking audit deliberately makes no gameplay claim about depth 2 as a policy. Nobody has run that comparison properly.