Sibling network on the 200-move panel

Start a network in a state where it is the reference search, exactly, then let it learn from a locked panel of positions in which every legal column was played forward two hundred moves — and see whether training moves it anywhere better.

It did not. In every held-out fold it moved away from the reference and ranked moves worse.

Why this experiment is unusually clean

Most learning failures in this repository have an easy excuse available. Either the training data only recorded the move that was actually played — the sibling trap — or the network started from random weights and might simply have landed in a bad place.

This design removes both excuses:

• Every legal column is labelled. The corpus is a locked panel of 477 positions drawn from 8 whole games, in which each sibling column was evaluated with a 200-move continuation. Nothing is extrapolated.
• The network starts as the reference policy. Its main output is a residual over depth-4 search, initialised to exactly zero. Before a single gradient step, the model's ranking is precisely fair depth 4's ranking. Any movement is training's own doing.
• Nothing is selected on the answer. One architecture and one training schedule are fixed before evaluation; the source states that there is no validation-driven choice of epoch, seed, feature, width or loss. Folds hold out one whole origin game at a time, and which game a position came from is never shown to the model.
• There is no way to cheat by playing. The program has no gameplay lane at all: it never creates a game, advances one, reads a seed, generates a panel or runs a screen.

How it works, step by step

1. Load the checksum-locked 477-record panel and verify its hash, its game count, and the number of records contributed by each of the 8 origin games.
2. Encode each position as sparse public features — the 49 cells, the visible next disc, the rise phase, the candidate column — plus summaries of the position after each candidate move.
3. Train five outputs at once: the residual over depth 4, and four auxiliary quantities — mean moves survived, numbered clears, reveals, and a 99th-percentile downside — with the ranking losses (pairwise and listwise) carrying most of the weight and the auxiliaries a small share.
4. Repeat for each fold, holding out one whole origin game, and compare against plain depth 4 on that game's positions: how often the top column matches, how often pairs are ordered correctly, and how much value is left on the table.

What happened, in plain English

The retained outcome is one sentence: it regressed depth-4 search on top-move accuracy, pairwise ordering, and leftover value in every origin fold.

That is a stronger negative than it first sounds. A model that begins as an exact copy of the best known policy, on data where every alternative was actually measured, with no chance to overfit a selection decision, still walked away from the reference and got worse. Whatever is wrong with learning to rank moves here, it is not a bad starting point and it is not missing sibling labels.

The technical record

• Status of record: rejected — task-record only (experiment index, "D4 and long-outcome research"): "it regressed D4 top-1, pairwise ranking, and regret in every origin fold."
• Read from the source rather than from a result: 477 expected records across 8 games from 0x3d6d0010, corpus SHA-256 beginning bfda8ae3; 5 heads, 64 hidden units, 48 epochs, batch of 32 roots, learning rate 0.0012, weight decay 1e-5, gradient-norm clip 3.0, loss weights pair 1.0 / list 0.75 / point 0.25 / auxiliary 0.12, one fixed network seed, pool reshuffled every epoch.
• audit-05 §2.3 quotes the source lines that make the zero-initialised residual exact — "the untrained model is exactly always-D4" — and §4 lists this as failure 16 of 17, one of the two experiments that actively refute the idea that these failures are optimisation or initialisation problems.
• Scale for the reader. Two other studies on this same 477-root panel are recorded with numbers, and they frame what "worse than depth 4" means here: a recomputation of exact search on the panel found depth 4 materially better than depth 1 on top-move accuracy, pairwise ranking, regret and every origin (completed — task-record only diagnostic), and a hindsight-based ranker charged for its information advantage scored 28.93% top-1 against fair depth 4's 38.16%, with pairwise 59.85% against 66.82% (ledger-recorded). Both live in terminal policy iteration.
• A caveat on the panel itself. The research roadmap records that this 477-root panel is "reusable diagnosis, not fresh model-selection evidence" (quoted in audit-05 §2.5). It has been read; a new model cannot be gated on it. §2.5 also records that the corpus file itself lived under /tmp and is gone from this machine.
• Source: d4-h200-sibling-nnue.cpp.

What this taught us, and what is still open

• Sibling coverage alone is not enough. The repository's most-cited failure mode was labelling only the move that was played. Here every move was labelled, and the ranking still regressed — which points at the labels rather than at their coverage: at one position, the true long-horizon values of seven columns are close together relative to the noise in any affordable measurement of them.
• Starting at the answer is not a shortcut. The zero residual is an elegant safety property — an untrained model that is exactly the reference — but it does not stop gradient descent from leaving it.
• Still open: whether a label with a smaller noise-to-gap ratio exists at all. The strongest signals measured anywhere in this repository are survival heads, not return heads, which is the argument audit-05 §5.3 makes for predicting per-rise survival instead of score.