gorge and dirac lockstep¶
dirac is trusted only as far as it plays what gorge plays. dirac/oracle runs
both engines on the same decks, seed and answers, and stops at the first
difference. The mechanics, the compared fields and the verdict table are on
Verification. This page is the map of the
guards around it.
flowchart LR
kf["kfuzz<br/>pool pairs, synthetic decks"] --> ls{"lockstep<br/>options, decision,<br/>projection"}
ls -->|"first difference"| rp["repro file"]
rp --> fix{"gorge wrong?"}
fix -->|"yes"| gf["fix gorge, then dirac"]
fix -->|"dirac wrong"| df["fix dirac"]
fix -->|"cannot yet"| rf["kgen refuses the card by name"]
ls -->|"limit reached"| cs["capacity stall, counted"]
rp --> pin["pinned game or repro test"]
The zero-divergence rule¶
A card dirac supports plays identically to gorge, decision by decision. gorge
leads. A gorge change that alters behaviour on a supported card fixes dirac in
the same change, or makes kgen refuse the card by name (kgen -refuse). There is
no list of tolerated divergences. scripts/dirac-gate.sh <base> holds this: it
runs kfuzz -absolute on the committed FDN and BLB slices on both surfaces, and
also checks the size, allocation and instruction budgets, the refused entries
and a short synthetic run.
Two kinds of difference are not divergences, and each is listed in
dirac/docs/DEVIATIONS.md:
a canonical-surface difference (the engines say the same thing in different words and
the oracle's adapter translates), and a declared capacity limit.
kfuzz and the two surfaces¶
kfuzz (dirac/oracle/cmd/kfuzz) plays seeded pairs of decks. The surface
chooses which options both sides offer.
Offers no plain cast that gorge's payment planner never prices, and no mana-ability options. It is the surface a hosted bot casts through.
Offers those casts too. It is the stricter surface.
make dirac-slices plays the first 1,500 pairs of the fdn and blb pools at
seed 1 on both surfaces with -absolute. The slices are committed under
dirac/oracle/testdata/slices. Build the tables first with make dirac-tables.
Beyond the slices, kfuzz -synth <pool> deals synthetic decks from a card list so
that cards the slices do not reach still get games. The flags are listed with
kfuzz -h. Other modes: -features also compares the network inputs
(dirac/feat against bots/policynet) at every decision, and -v2 transcript
checks the SpellBench v2 adapter against gorge's.
Verdicts¶
Every game that started ends in one of these:
| Verdict | Meaning |
|---|---|
| clean | agreed to the end, or to a turn or decision cap |
| capacity | dirac hit a declared limit; agreed up to there |
| unmodelled | dirac stopped at something gorge does and it does not |
| divergent | the engines differed; a panic counts |
| hang | no result inside -game-timeout |
A pair holding a card or combination dirac does not support is not played, and is
counted as refused-card or refused-pair. -absolute fails on any divergence,
hang, unmodelled stall or error, and when too few games were played.
Repro files¶
A divergence is written with -out DIR as a portable repro: the two decks by card
name, the seed, the run knobs and the canonical actions, shrunk by -shrink N.
kfuzz -repro FILE replays it with no deck pool. The fixed ones are kept under
dirac/oracle/testdata/*.repro.json and replayed by tests, so a fixed divergence
cannot return silently.
Pinned games¶
A pinned game is a recorded game: two decklists by card name, a seed and the
canonical actions, under dirac/oracle/testdata/pins. Each is played in lockstep.
It must reach the conditions its test names, such as a card entering or a trigger
resolving, and stay clean. Because it does not depend on how a synthetic pool
deals its pairs, lowering or refusing another card cannot move it. A game that no
longer reaches its condition fails, so a fix cannot hide by changing the path.
Mutation tests¶
A clean lockstep run proves nothing if the comparison cannot fail. Many
class tests have a mutation twin (the ...Mutation tests in dirac/oracle):
the same games with one behaviour deliberately made wrong, which must diverge.
TestAbilitycastMutation fails if its mutated games do not. The class test shows
a feature agrees. The mutation test shows the oracle would have seen it differ.
Refused entries¶
When a card lowers and still plays wrong, it is refused by name. Each entry is a
JSON file in dirac/support/refused/ with the card, a class (such as
unmodelled-...), a reason and a repro pair. The rules:
- A card is refused only while it diverges.
TestRefusedEntriesStillDivergefails if an entry's card no longer diverges anywhere in its pool, and the entry is deleted. - A card cannot leave the evidence lists in
dirac/support/evidence/unless the change refuses it.dirac-gate.shstep 0 checks this. - Refusals are generated into dirac's
support page,
held current by
TestSupportDocIsCurrent.
Capacity stalls¶
dirac has fixed capacities: the battlefield, objects, pending triggers, the stack,
and some rings that track the turn. A game that outgrows one stops as a named
stall. It is a declared limit, counted per table, printed in the summary line, and
not compared past that point. It earns coverage up to where it stopped. Raise a
limit when realistic play hits it. A limit only random play hits is evaluated,
not raised by default. The list is in
DEVIATIONS.md.
dirac's CR coverage¶
dirac/crcoverage is a rule-by-rule table with the code that decides each rule,
held to the code by TestCRCoverageIsCurrent. A row marked inherits gorge
copies gorge's behaviour, so gorge's table
decides it.
What lockstep does not show¶
It shows agreement, not correctness. dirac lowers gorge's compiled IR, so a shared reading of the rules is shared by both sides. That is why gorge is also checked against engines that do not share its source: see Engine comparison.