Evaluating a board, and the sibling trap

A look-ahead search ends at boards where the game is not over. It needs an opinion about each one: how promising is this? That opinion is the leaf evaluator, and almost every learning experiment in this repository is an attempt to replace the hand-written one with something learned from data.

Most of those attempts failed in the same way. This page explains the failure in plain language, because it is the key to reading the research pages — and to judging any proposal to "just train a bigger model."

The job is to rank, not to predict

When a policy plays, it does not need to know how many points a board is worth. It needs to know which of up to seven sibling moves — the seven columns available right now — is best. A perfect ranking with wrong absolute numbers plays perfectly. A model with tiny average error that gets the order of two siblings wrong plays the wrong move.

That distinction sounds academic until you look at how training data is made.

How the data was collected, and what it missed

The obvious way to get training data is to play games and write down what happened: this board, this move, this outcome. Do that for thousands of games and you have a large dataset. But look at what it contains for any one position:

The model trains on the boards that were played and gets very good at predicting their outcomes. Then it is deployed, shown seven siblings, and asked to rank them. Six of the seven are boards whose futures were never measured. Its answer for those is an extrapolation from boards that looked somewhat similar — and if the behaviour policy had a consistent style (they all do), the unplayed siblings are systematically the kinds of boards it never chose, so the model is systematically worst exactly where it is being asked to decide.

The repository's status summary names this directly:

A repeated problem was sibling extrapolation: a model learned the outcome of the action that was played, then deployment asked it to choose among several actions it had not observed equally well. Low value error on visited states did not guarantee good root-action ranking.

Measured, not supposed

This is not a hypothetical. The latest completed work built a locked panel of 477 real positions and evaluated every legal sibling at each one, so that a model's ranking could be checked against a reference. In plain terms: the best learned ranker at the time picked the same top move as fair D4 about 29% of the time, against D4's own 38% agreement with the gold standard, and ordered pairs of siblings correctly about 60% of the time against D4's 67%. It lost on both counts. A second, more conservative variant barely disagreed with D4 at all and its few disagreements were not reliably improvements.

The technical record

From the research status summary §7: a martingale-dual H12 ranker scored 28.93% vs 38.16% top-1 accuracy and 59.85% vs 66.82% pairwise accuracy against fair D4 on the 477-root panel, with higher normalised regret. A regenerative policy-iteration variant overrode D4 on 11 roots; six overrides helped, confidence bounds were negative, and five of eight origins did not regress. Neither justified a gameplay run.

What would fix it

The fix is conceptually simple and practically expensive: label every sibling. For each training position, evaluate all legal columns — under the same random futures, so the comparison between them is fair — and record all of them. A dataset built that way is called successor-closed here: every successor the model will be asked about at play time is present in training.

That is precisely what the repository's registered afterstate theory proposes. In plain words it says: train a single evaluator only on successor-closed data, with every sibling labelled under aligned chance scenarios and a fixed continuation policy, and it will rank root moves at least as well as fair D4 on games it has never seen.

It was built and tested, and as tested it did not work. Four pilot experiments are on the record. The corpus pipeline did exactly what it promised — every legal sibling labelled, under chance scenarios shared across siblings — but the labels themselves turned out to be noisy: with a handful of imagined futures per sibling, re-running the same measurement on a different half of the scenarios produced a substantially different ranking. Raising the number of aligned futures to 256 fixed the stability problem, and the trained model still did not rank moves better than fair D4. A final, more conservative experiment let the model overrule the search only where the search's own top two moves were nearly tied; that came closer than anything before it — it improved on fair D4 in both halves of the data — but the improvement in the second half was smaller than the margin fixed in advance, so it is recorded as a valid run with a failed gate, not as a success.

The honest reading is that the data shape was fixed and the problem moved: what limits the model now is the quality of its labels and its own training, not the sibling gap. The result record itself names an undertrained checkpoint (11 of 20 planned epochs) and a weak teacher as the likely bottleneck. That is a different, more tractable complaint than the one this page opened with — and it is where the more-computation question picks up.