Skip to content

The AZ loop, and why it went flat

2026-10-08 → 10-21

The AlphaZero-style loop trains the d64 MLP on its own search's visit counts, generation by generation, and gates each generation in play. This page covers the hosted lineage (az-c3, az-c4, az-c5), the from-scratch az-c6 loop to g7, the kernel-versus-gorge generation test (loop2), and the 2026-10-18 → 10-21 work that showed the loop had stopped learning, and why. The f84x line that followed is on Net width and Policy target and react.

Verdict

  • az-c4 (C4 of the census-on series) was adopted as the hosted bot; C5 improved on it only within noise (0.528 [48.8, 56.8]).
  • The az-c6 loop plateaued at g6–g7 (vs prev 0.490, 0.500). More generations with the same recipe, on either engine, did not move it.
  • Adopted from the diagnosis: cpuct 0.5 with root LCB and :defer=0.1 at play, and more sims at play time. The training recipe moved to the f84x from-scratch loop and the policy-target and react A/Bs.

Records lost

The c-series training tree (its plan, the loop data and eval summaries), the az-c5 run and the stage-0 run no longer exist. The tree was "rebuilt from scratch (2026-10-16)", and they are not in the 2026-10-25 archive. The c-series numbers below come from the design notes and the retrain write-up that recorded them, both kept privately.

1. Question

Can search-labelled self-play make the hosted search bot stronger, generation after generation, and what stops it?

2. The loop

flowchart LR
  SP["self-play<br/>az:sims=25, :explore,<br/>closed list"] -->|every 4th position| REC["visit records"]
  REC --> PACK["pack"]
  PACK --> TR["train mlp64<br/>cq policy target,<br/>blended value target"]
  TR --> NET[".dzgw"]
  NET --> GATE{"gate: 600 games<br/>vs prev and vs sb-tactical"}
  GATE -->|pass| SP
  GATE -->|two fails| STOP["stop"]

The census-on c-series loop: self-play at 25 sims per move with the closed list, opponent nodes, autopay and top-k 6 on the train-split decks, with every 4th position recorded; the net is mlp64 (d 64, ff 128, 1 layer, card_d 32, head 64). Gates are 300 pairs (600 games) at 100 sims, seed 78: vs full search with no net, vs sb-tactical, vs the previous generation. A generation passes at ≥ 50% vs the previous and an sb-tactical score no lower than the previous one's; two consecutive fails stop the series. Decks: DraftZero FDN Limited, 28,366 train and 3,150 eval, London mulligan up to 2.

3. The hosted lineage: c3, c4, c5

c3, open list (2026-10-09). Each row is 600 paired games, 100 sims, FDN eval decks, seed 78, Wilson 95%.

c3 against W-L Score 95% CI ms per searched decision
sb-tactical 378-222 63.0% [59.1, 66.8] 314
no-net search 335-265 55.8% [51.8, 59.8] 202
c2 307-293 51.2% [47.2, 55.1] 344

Earlier generations vs sb-tactical: c1 58.5% [54.5, 62.4], c2 61.7% [57.7, 65.5].

Closing the opponent's list (2026-10-13 → 10-14). The search had redealt hidden cards from both real lists. Old c3 scored 61.2% vs sb-tactical with a closed list at play time against 63.0% open, so the list was worth at most about 2 points. The census-on retrain, every row 600 games:

Gen Self-play games/s Positions vs full search vs sb-tactical vs prev vs census-off C3 Gate
C1 17.6 777,838 54.7% 59.8% — 46.8% first
C2 11.8 494,113 58.1% 63.8% 51.7% 52.2% pass
C3 13.9 562,420 59.3% 65.8% 51.7% 52.7% pass
C4 13.5 524,846 60.8% 67.8% 53.7% 55.2% pass, best
C5 12.8 523,750 61.8% 67.5% 49.8% 56.0% fail 1
C6 15.8 608,750 60.7% 66.7% 48.7% 56.5% fail 2, stopped

C4 landed as the hosted bot az-c4 (gorge 9ba4de165, 2026-10-14). Its gates with Wilson intervals: vs sb-tactical 407-193, 67.8% [64.0, 71.4]; vs the same closed search without a net 365-235, 60.8% [56.9, 64.7]; vs C3 322-278, 53.7% [49.7, 57.6].

c5 (2026-10-15). Warm start from az-c4 after the payment fixes, four generations of self-play (32,000 / 28,000 / 32,000 / 32,000 games). 600-game gates:

Net vs az-c4 vs sb-tactical
az-c4 on this build — 0.690 (414-186) [65.2, 72.6]; 0.663 before the payment fixes
g1 0.522 (313-287) 0.712 (427-173)
g2 0.503 (302-298) 0.710 (426-174)
g3 0.517 (310-290) 0.705 (423-177)
g4 = C5 0.528 (317-283) [48.8, 56.8] 0.723 (434-166) [68.6, 75.8]

The design note observes that about 1,200 paired games would be needed to put the interval above 50%.

4. az-c6 from scratch, g1–g7

2026-10-16 → 10-17. g1 plays 100 sims with exploration and no net; later generations play 25 sims with the previous net, every 4th decision recorded. Training uses the cq policy target and a blended value target, starting from the previous best, with every pack so far in the replay buffer. Gates: 150 pairs (300 games), seed 78, eval split. No CI was printed; at 300 games it is about ±0.05.

The FDN self-play loop by generation census-on c-series vs sb-tactical (600 games) az-c6 vs sb-tactical (300 games) az-c6 vs the previous generation (300 games) 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 g1 g2 g3 g4 g5 g6 g7 generation C1 vs sb-tactical: 0.598 [0.559, 0.637], 600 games C2 vs sb-tactical: 0.638 [0.600, 0.676], 600 games C3 vs sb-tactical: 0.658 [0.620, 0.696], 600 games C4 vs sb-tactical: 0.678 [0.641, 0.715], 600 games C5 vs sb-tactical: 0.675 [0.638, 0.712], 600 games C6 vs sb-tactical: 0.667 [0.629, 0.705], 600 games az-c6 g1 vs sb-tactical: 0.600 [0.545, 0.655], 300 games az-c6 g2 vs sb-tactical: 0.657 [0.603, 0.711], 300 games az-c6 g3 vs sb-tactical: 0.680 [0.627, 0.733], 300 games az-c6 g4 vs sb-tactical: 0.673 [0.620, 0.726], 300 games az-c6 g5 vs sb-tactical: 0.687 [0.635, 0.739], 300 games az-c6 g6 vs sb-tactical: 0.727 [0.677, 0.777], 300 games az-c6 g7 vs sb-tactical: 0.720 [0.669, 0.771], 300 games az-c6 g2 vs g1: 0.560 [0.504, 0.616], 300 games az-c6 g3 vs g2: 0.520 [0.463, 0.577], 300 games az-c6 g4 vs g3: 0.510 [0.453, 0.567], 300 games az-c6 g5 vs g4: 0.527 [0.471, 0.583], 300 games az-c6 g6 vs g5: 0.490 [0.433, 0.547], 300 games az-c6 g7 vs g6: 0.500 [0.443, 0.557], 300 games
Gen vs sb-tactical vs prev Self-play games (rate)
g1 180-120 = 0.600 — 8000 (3.9 games/s)
g2 197-103 = 0.657 168-132 = 0.560 16000 (6.4)
g3 204-96 = 0.680 156-144 = 0.520 32000 (12.2)
g4 202-98 = 0.673 153-147 = 0.510 32000 (12.7)
g5 206-94 = 0.687 158-142 = 0.527 32000 (12.5)
g6 218-82 = 0.727 147-153 = 0.490 28000 (10.8)
g7 216-84 = 0.720 150-150 = 0.500 24000 (9.2)

5. Kernel or gorge for self-play

A one-hour test and loop2, 2026-10-18.

One hour of the c6 recipe on each engine from g7, same budget each; both corpora trained from g7 and gated against g7.

Engine Games Visit records Gate vs g7, kernel, 1,500 pairs Gate vs g7, gorge, 150 pairs
gorge ~4,920 94,083 0.501 ±0.018 0.490 ±0.057
dirac kernel 132,298 2,494,899 0.536 ±0.018 0.527 ±0.056

The g7-vs-g7 control scored 0.508 ±0.025. vs sb-tactical (300 games): g7 0.720 ±0.051, kernel-hour net 0.747 ±0.049, gorge-hour net 0.713 ±0.051.

Two lines from g7, three generations each. The kernel line (k) self-plays at 2000 sims per move on the dirac kernel; the mainline (g) at 25 sims in gorge. Training starts from the previous net on az-c6's packs g1–g7 plus the line's own; the net has 601,346 parameters. gorge gates are 300 games (±0.05), kernel gates 3,000 games (±0.018).

Gen Self-play vs sb-tactical (gorge) vs prev (gorge) vs prev (kernel)
k8 1464 games, 26,684 records 0.713 ±0.051 0.483 ±0.057 0.514 ±0.018
k9 1375 games, 25,630 records 0.727 ±0.050 0.497 ±0.057 0.507 ±0.018
k10 1402 games, 25,847 records 0.700 ±0.052 0.493 ±0.057 0.500 ±0.018
g8 163,349 records 0.703 ±0.052 0.487 ±0.057 —
g9 133,241 records 0.730 ±0.050 0.500 ±0.057 —
g10 128,010 records 0.717 ±0.051 0.483 ±0.057 —

Neither line moved. One generation's records were 0.8% (k) or 4.6% (g) of the replay buffer, which pinned the net to g7; sims=2000 bought 18× fewer games.

6. The diagnosis

2026-10-18 → 10-21.

Search beats training. On the kernel's scale (3,000 games vs the kernel's sb-tactical port, ±0.017), g7 with more search beat every trained arm:

  • g7 at 100 sims or fewer
  • a trained arm at 100 sims
  • g7 with more search
Arm vs sb-tactical
g7, sims 25 0.564
g7, sims 100 (reference) 0.618
g7, sims 400 0.681
g7, sims 1600 (1,000 games, ±0.028) 0.702
loop2 g10 0.625
window cut to g9+g10 0.627
ff512 0.629
retrained from scratch 0.634
loop2 k10 0.638
from scratch, 10 epochs 0.640
the one-hour kernel net 0.644

These are not comparable with the gorge sb-tactical numbers above: the same g7 scores 0.618 here and 0.720 in gorge, because the kernel's fallback bot at the time was a weaker stand-in. With gorge's heuristic bot ported natively, g7 scores 0.708 ±0.016 on the kernel.

The xp series (2026-10-19 → 10-21) then tested search knobs and a new training recipe on the kernel:

  • cpuct 0.5 with root LCB, sims 100: +5.0 ±1.4 points vs sb-tactical (3,000 games per arm); cpuct 0.5 alone +3.0 ±1.3; in gorge +3.63 ±1.19.
  • sims 400 → 1600: +3.8 ±1.3; 1600 → 6400: +1.5 ±2.0.
  • :defer=0.1: +0.1 ±1.2 at sims 400, and own-upkeep instant casts fell from 50% to 0.3%. Adopted.
  • Knowing the opponent's deck list at sims 1600: +0.6 ±1.2; knowing the hand: +2.8 ±1.2. Belief work parked.
  • The P2 arm (cpuct 0.5, every decision recorded, visit targets with a root-value blend, a short replay window): 0.556 vs g7 at iteration 4 with sims 200, against 0.524 at iteration 4 for the old c6 recipe (3,000 games per point, ±0.018).
  • Best play: g7 at sims 400 with cpuct 0.5 and root LCB, 0.784 ±0.015 vs sb-tactical.

KL(visits‖prior). The trained prior already agreed with the search that trained it, so the policy target carried almost no new information:

Measurement KL mean KL median
kclair i1 pack 0.0949 0.0434
kfresh i1 (closed-list control) 0.0842 0.0400
kfresh i2 0.0905 0.0433
sploop pack_g8, 40k states (2026-10-24) 0.076 —

On the g8 pack the top-1 visit matched the prior's top-1 87% of the time, and 39% of states had a root Q span below 0.02. The packs behind the 2026-10-21 reading (kvb0.5 i5, kvb0 i2, az-c6 g7) and its script survive, but its output was not saved; see records missing.

Reading

The visits at sims=25–100 and cpuct 0.5 with at most six candidates rarely disagree with the prior, and the value target is half the search's own root value, so the loop distils itself. The xp series showed that search knobs paid more than any training arm.