One dial was controlling two different pieces of luck
completedevidence: development-tier exploratory findingpublic informationThe reference search drew "which disc comes next" and "what the hidden discs turn out to be" from the same counter, so it never imagined the two happening in different combinations; giving them separate draws is worth about as much as a whole extra move of look-ahead.
The reference search drew "which disc comes next" and "what the hidden discs turn out to be" from the same counter, so it never imagined the two happening in different combinations. Giving them separate draws is worth about as much as a whole extra move of look-ahead.
The intuition: one move, two futures
When a move causes a clear next to a gray disc, the gray disc's hidden number is revealed — and what that number turns out to be can change everything about the move. Below is one real position from the rules engine. The next disc is a , and it goes into the second column from the left.
The 5 lands on top of the second column, which makes the 4 beneath it the fourth disc in a vertical run of four, so the 4 pops. That clear hits the cracked gray disc under it for the second time and shows its number. If it is a 3, the 3 is immediately in a run of three and pops too, and a four-wave chain runs: 379 points, and the board ends the move with three fewer discs on it than it started, even after adding one.
Same board, same drop. The 4 pops and the gray disc reveals — but as a 4, which sits in a run of three and does nothing. One wave, 7 points. Both outcomes are the engine's; the only difference is a number no player can see.
A search cannot know which of those it will get, so it has to imagine several possibilities and average them. That is a chance node, and the number of possibilities it imagines is the search's chance samples (the repository calls them strata; the reference uses five, and a stronger variant uses seven).
Here is the defect this approach found. Every chance node has two independent pieces of luck — the number revealed under the gray disc, and which disc the game deals next — and the reference search indexed both with the same counter. Sample 1 always paired a particular reveal with a particular next disc; sample 2 always paired the next pair; and so on. With seven samples the search saw seven of the 49 possible combinations, and the seven it saw were perfectly correlated. A move whose value depends on the combination could not be evaluated correctly.
How the fix works
- Split the dial in two. Instead of one setting used for both quantities,
the search takes a number of disc samples
Nand a number of reveal samplesM, and runsN × Mscenarios per chance node. The reveal draws are spaced through the unit interval so that each disc branch still sees a properly spread reveal sample. - Prove it is the same policy when
M = 1. At one reveal sample the two loops collapse to exactly the old one — the same draws in the same order with the same divisor — so the existing arms become special cases rather than things that merely resemble them. - Count what the samples actually cover. A separate script re-implements the engine's own quadrature in Python and measures, over 20,000 synthetic chance nodes, what fraction of the joint outcomes get any weight at all. This is what turns "more samples" into a specific claim about which part of the chance node was broken.
- Set the work budget per configuration. The reference search carries a work limit sized for its own branching factor; leaving it in place while widening the chance node silently degrades a four-move search into a three-move one and reports the result as a chance-sampling finding. Every arm here declares its own limit and the binary refuses to run a cohort without one.
- Play whole games, 64 of them per arm, paired on the same seeds as the earlier chance-sampling study so the comparison is direct.
What happened
The fix pays, and it pays in proportion to how much of the chance node it covers. Holding the disc side exact at seven samples and the look-ahead at three moves, raising the reveal samples from one to three to six raised the mean score from 312,327 to 337,306 to 376,442 — and the fraction of the disc-and-reveal combinations receiving any weight went 14.3% → 42.9% → 85.7% in lockstep. Score, lifetime, clears per move, reveals per move and (downward) board occupancy all ordered correctly across the three settings.
The endpoint clears its statistical bar; the individual steps do not. Six reveal samples beat one by +64,116 points with a one-sided 95% bootstrap lower bound of +7,475, winning 36 of 64 paired games. Each single step along the way is inside the noise on 64 games. The claim therefore rests on the ordering of five quantities across three settings plus the significant endpoint, not on any one pairwise test.
The headline is against the reference policy. A three-move search with six reveal samples beats the frozen four-move reference by +79,115 points (95% lower bound +30,242), and is statistically indistinguishable from the strongest previously measured arm — the four-move search with seven chance samples — while doing 86% of its work. So the earlier conclusion that "depth needs an exact chance estimator" is only half of a symmetric statement: at a fixed budget, a fourth move of look-ahead and a decorrelated chance estimate buy roughly the same thing, and the chance estimate is slightly cheaper.
The mechanism moved, which is the check that mattered. From one reveal sample to six, numbered clears rose by 0.0598 per move and gray-disc reveals by 0.0422, with mean occupancy falling from 23.88 to 23.49 of 49 cells. The six-sample three-move arm reaches exactly the four-move reference's reveal rate of 1.1423 and comes within 0.002 of its clear rate — a shallower search reproducing a deeper search's flow profile.
It is bought with compute, at about the same exchange rate as depth. Going from one reveal sample to six costs 27.1 times the logical work per move. Score per unit of work falls monotonically as the reveal samples rise, exactly as it does with depth. What is new is not free strength; it is a second axis on the frontier that happens to be available at a shallower and cheaper depth.
None of it reaches the requirement. The best arm here sustains 2.0447 clears and 1.1423 reveals per move against the 2.400 and 1.400 that indefinite survival needs. Every game still filled its board and ended.
The record: cohorts, gates and the arms that were not finished
Source: finding-09-reveal-sampling,
a retained exploratory finding at the repository's development evidence tier
for the gameplay arms and pilot for the runtime projections, measured in this
checkout on 2026-08-20. It has no row in the
experiment index and no entry in the
ledger. Run validity: valid. Outcome: pass, with
the caveat that the effect needs six reveal samples to clear noise on 64 games.
Cohort. The fixed shared 64-game evaluation cohort 0xa51d1000–0xa51d103f,
2,000-move cap, corrected 17,000-point Hardcore scoring, 0 censored games and 0
score-decomposition identity violations in every arm. These seeds were already
read by the earlier chance-sampling study and are development data permanently,
so nothing here can serve as confirmation evidence.
Arms (depth 3 unless stated; N disc samples × M reveal samples):
| arm | mean | median | moves | clears/move | reveals/move | occupied | work/move |
|---|---|---|---|---|---|---|---|
| N=5, M=1 (frozen chance node) | 305,051 | 259,100 | 89.84 | 1.9638 | 1.0861 | 23.88 | 54,429 |
| N=5, M=3 (control) | 312,556 | 281,704 | 92.00 | 1.9868 | 1.1111 | 23.54 | 368,518 |
| N=7, M=1 (seven-sample arm) | 312,327 | 267,279 | 92.27 | 1.9849 | 1.1001 | 23.88 | 156,834 |
| N=7, M=3 | 337,306 | 285,023 | 98.70 | 2.0033 | 1.1111 | 23.81 | 1,045,719 |
| N=7, M=6 | 376,442 | 322,859 | 109.45 | 2.0447 | 1.1423 | 23.49 | 4,244,020 |
| depth 4, N=5, M=1 (reference) | 297,327 | 260,415 | 87.16 | 1.9489 | 1.0697 | 24.29 | 1,296,034 |
| depth 4, N=7, M=1 | 398,498 | 344,630 | 114.66 | 2.0571 | 1.1549 | 23.15 | 4,956,614 |
The two depth-4 rows are the earlier study's arms on the same cohort, reproduced for comparison and not re-run.
Paired deltas, one-sided 95% lower bounds by percentile bootstrap over whole games, 20,000 resamples: (7,6) − (7,1) = +64,116 [+7,475], 36–0–28; (7,6) − (5,1) = +71,391 [+13,603]; (7,6) − depth-4 (5,1) = +79,115 [+30,242], 35–0–29; (7,6) − depth-4 (7,1) = −22,056 [−89,867], 30–0–34, read as a tie. The two intermediate steps, (7,1)→(7,3) at +24,980 and (7,3)→(7,6) at +39,136, do not clear zero.
Coverage (20,000 synthetic chance nodes): at seven samples every individual marginal is already exact — seven strata over seven equally likely values. The residual error is in the joint. Reveal samples raise the (disc, reveal) joint coverage 14.3% → 42.9% → 85.7% → 100% at M = 3, 6, 12, but the reveal-by-reveal joint only 14.3% → 26.2%, because all reveals inside one scenario still share the sample index. The measured strength ordering tracks the first column and not the second, which is a specific, falsifiable attribution of which correlation was the binding defect.
CHECK gates, all before any gameplay: the new search reproduces the frozen
reference's chosen column over 50 moves; it reproduces the existing
parameterised search's column and logical work over 562 decisions across four
configurations with 0 mismatches; and both M = 1 arms reproduce the existing
binary's own 64-game artifacts field-for-field, including summed logical work to
the unit. Work bounds were computed per configuration and verified never to
bind: 0 of 30,865 decisions completed below target depth, busiest single
decision at 77% of its bound.
Recorded rather than omitted: a depth-4 arm with two reveal samples was launched on the shared cohort and killed by the runtime after about an hour with no games written — a partial run, not a censored result, and it appears in no table. It was authorised by a matched pilot rather than guessed. It is the highest-value open arm, because it is the only way to learn whether the two axes compound rather than substitute. A three-move arm with twelve reveal samples, the configuration that takes joint coverage to 100%, was a deliberate no-run at roughly 5.7 hours of the machine.
Limitations the finding states about itself: depth 3 only for the completed arms; the individual dose steps are not separately significant; reveal samples are mostly a decoupling knob and barely touch the reveal-by-reveal joint; 64 paired games with a score standard deviation of 55–62% of the mean; one already-read cohort, so replication on a fresh block is owed; no fixed-time comparison, and the machine carried load averages of 20–55 throughout; and the cache size was raised above the frozen default for the widest arm, a declared deviation that cannot change a decision because the work bound assumes no cache at all.
What this taught us, and what is still open
- The defect has a precise name, which is what makes it a finding rather than a tuning result: it was not "too few samples" and not "the next-disc distribution is biased" — at seven samples every individual distribution is already exact. It was that the next disc and the reveals were the same random variable in disguise. That is a two-line change to how one index is computed.
- Depth and chance quality are exchangeable, not a hierarchy. Two ways of spending the same budget, with the cheaper one available at a shallower depth.
- Still open: whether the two axes compound when used together. The arm that would answer it was killed by the runtime and never produced a game.
- Still open: the reveal-by-reveal correlation. A design giving each scenario its own independent stream of reveal draws would attack it directly and has not been built.
- Context for how much this is worth. +64,116 points on a mean near 376,000 is a real improvement and is nowhere near the target. The flow ceiling page has the measurement of how far there is to go.
Source files
README.mdxchance-coverage.pycompare.pypool.pysearch.cppstats.py