How-to guides¶
Task-sized guides for building a follower. They assume you've done the quickstart and keep the protocol open beside them.
Grow the runner in milestones¶
Each stage below is useful on its own, and anything a stage can't do yet becomes a harness row
rather than a false finding. So you can run real sweeps from the first stage on.
| Stage | Build | Corpus | Done when |
|---|---|---|---|
| 0. Card check | --preflight |
oracle/xengine/pools/fdn-commons.txt |
It refuses what your engine knows it can't play, and nothing else |
| 1. Setup | Place the board, reach the setup checkpoint, write one snapshot; harness for every step |
oraclediff gen -level A on FDN |
The "setup" snapshot agrees on nearly every item |
| 2. Cast and resolve | cast, activate, resolve, pass; targets, modes, X, yes/no, choose-n from the script |
The same, then all FDN | Every row agrees, or carries a reason you understand |
| 3. Windows | pass_to, attack, block, attach; combat prompts |
xengine play -pool … -windows |
decision_shape is a small share of sent windows |
| 4. Tapes | Whole games from turn 1: real turn structure, draws, cleanup | The same play directories | Tapes run to the point where the lowering cut them |
Widen the corpus only when the current one is quiet: FDN commons (fdn-commons.txt, 196 cards), then
all of FDN (fdn.txt, 517), then other sets with -set.
Place a board without entry effects¶
Setup is the part most engines lack a public API for. The contract (the setup table) is that placed permanents arrive as if they had always been there: no ETB triggers, no "enters tapped" or "enters with counters" replacements, no summoning sickness, no "this turn" history.
What usually works:
- A debug or editor API, if your engine has one for a deck builder or test harness. Check that it really skips the entry pipeline. A function that "adds a card to the battlefield" by moving it there usually runs replacements.
- A small placement hook in the engine: put card X in zone Z for controller C with these flags, and touch nothing else. In the Forge driver this is about 500 lines because Forge has no such hook. In an engine you control it's a few dozen lines, and worth sending upstream.
- Counters are added on top. A planeswalker placed with
{"LOYALTY": 1}ends up with its printed loyalty plus one. Tokens are the exception: theircountersare the complete set.
Then clear any per-turn bookkeeping the placement left behind, set turn 1 with p0 active, and run the
engine forward from the untap step to p0's first main-phase priority. Apply tapped_after_untap there,
and take the "setup" snapshot.
Test it against gorge.jsonl first: compare only checkpoint 0 of each item until it agrees.
Drive priority exactly as gorge does¶
Most early disagreements are about priority, not cards. gorge follows the CR:
- One
passstep is one priority pass by the named seat. Your engine must not auto-pass for either player, and must not stop anywhere else. If a different seat holds priority when apassarrives, the sequences have drifted. Stop with a harness row rather than guessing. - Priority is given in every step the CR gives it: upkeep, draw, beginning of combat, each combat step, end step, and both main phases. An engine that skips a step, like Forge skipping combat damage when no creature is left in combat, needs a registry quirk so the classifier knows.
resolvepasses until the stack is empty and the active player holds priority, never stopping at a cleanup priority.- The checkpoint after a cast is when priority comes back to the caster with the spell on the stack, not when it resolves.
If your engine gives priority somewhere gorge doesn't, report it as a harness row with a clear message the first time it happens, and tell us. It may be a gorge bug.
Answer decisions strictly¶
Answer every prompt from gorge_decisions, and never let the engine's AI or a default fill a gap.
- A prompt with one legal answer can be answered automatically. Note it in your row's
notes. - A prompt with several answers and no recorded decision is a strict miss. Set
strict_misstoMissing <what> answer at step <i>and stop. The item is classeddecision_shape. - A recorded decision you never used goes in
leftover. - Don't add a "loose" retry in the first version. The Forge driver re-runs a missed item with Forge's AI answering, but loose rows are much harder to interpret.
A silent default is worse than a miss: it turns into a diverge that someone has to read.
Match an ability to gorge's index¶
activate steps carry gorge's ability_index, an index into gorge's compiled card.
- If your engine reads Forge scripts (ManaBrew does),
abilities[step]is the exact script line. Match it against your parsed abilities. - Otherwise, use
neutral.ability[step]. Find the abilities of the sameclasson the face, take the one atordinal, and check itscostandtextagainst yours. If they don't match, stop with a harness row such asharness: ability_unmatched at step 3. Don't fall back to "the first one". - Keyword abilities (Equip, Crew, …) have
keywordinstead of an ordinal: match by keyword and cost. - Basic land mana is
intrinsic_mana, withproducesnaming the colour.
Pay from the pool¶
The harness never asks your engine to pick lands to tap. Every mana source gorge used is its own
activate step before the spell. So:
- Activate mana abilities as their own steps. Each completes at once, with no stack.
- When the spell or ability is cast, pay its cost from the floating pool only, spending the units in
payment[i]first. - If the engine's own auto-payment would tap anything, turn it off for the run.
If your driver spends payment[i] exactly, set ConsumesPayment: true. Then a pool split that differs
after a cast is reported as a real difference.
Write snapshots gorge can compare¶
A checklist, in the order mistakes usually show up:
- Step names are gorge's (
main1,begin-combat,combat-damage, …), and both damage steps arecombat-damage. - Colours are WUBRG letters in that order, and colourless is
"". The pool is WUBRGC letters. handandexileare sorted,graveyardkeeps its order with the oldest card first, andlibrary_topis the top five, top first.typesholds card types, supertypes and subtypes together, sorted.ptis only set for creatures, after every continuous effect has applied. Recompute layers after any setup change.attached_tois the host's current name, orpNfor a player.- A stack entry's
sourceis the card's name, not an id. - Counter kinds use gorge's names (
P1P1, not+1/+1). - Card names use gorge's spellings: map yours on the way out.
Debug one item¶
Every finding gets a directory under <diff out>/finds/<class>/<id>/:
| File | What it is |
|---|---|
REPRO.md |
The commands to re-export, re-run and re-diff just this item |
input.jsonl |
The item as one line, cut after the step that differed, ready for follower-export |
scenario.json |
The same item, pretty-printed for reading |
gorge.json |
gorge's result: snapshots, decisions and a transcript of the game |
<engine>.json |
Your row |
verdict.json |
The class, checkpoint, field and both values |
REPRO.md ends with an xengine diff … -only <id> command, which classifies one re-run row without
re-running the whole chunk. To see why gorge did something, read the transcript in gorge.json.
Handle shuffles and randomness¶
You don't need to make your engine's shuffles match gorge's:
- A tape stops at the first random choice: a shuffle, a coin flip, a random discard.
- A window whose turn shuffles a library carries
ignore: ["library_top"], so library order isn't compared. - A hand or library difference after a recorded random draw is classed
rng, notdiverge.
Do seed your own randomness from the request, so reruns are identical. If your engine would consult randomness for something gorge didn't (a coin flip gorge's script doesn't have, say), stop with a harness row. Cards whose effects need random outcomes are good candidates to refuse in the preflight.
Pin your engine and card data¶
A difference in card data looks exactly like a rules difference, so both sides pin:
- gorge-compliance pins a gorge commit (
GORGE_PIN), and gorge pins the Forge card scripts it compiles (FORGE_REFin gorge's Makefile). - Pin your engine to a commit, and your card data to the revision built from it. Your setup script should refuse to run if they don't match.
- If your engine reads Forge scripts, build its card data from gorge's
FORGE_REF, not from whatever Forge version your engine ships with. The Forge runner checks that its card-script trees are byte-identical toFORGE_REF's before every run (scripts/forge-oracle-setup.sh --check-pin).
Bump on purpose, never to "latest" automatically, and re-run the same corpus after a bump to see what it changed.
Run your engine beside Forge and XMage¶
The sweep takes a list of followers:
XENGINE_ROOT=/tmp/xe XENGINE_FOLLOWERS='forge xmage toy' \
XENGINE_POOL=oracle/xengine/pools/fdn-commons.txt \
scripts/xengine-sweep.sh --sets FDN --chunks 4 --games 50
| Variable | Meaning |
|---|---|
XENGINE_FOLLOWERS |
Space-separated registry names. Default forge xmage |
XENGINE_ROOT |
Output root, outside the repository |
XENGINE_POOL |
A card list to draw decks from (passed to play -pool) |
XENGINE_POOL_FILTER |
Card names to leave out of the pool |
XENGINE_<NAME>_RUNNER |
Override your runner's path |
XENGINE_<NAME>_MEM |
Memory for your runner's heavy lock. Default 12G |
XENGINE_<NAME>_WAIT |
Seconds to wait for the heavy lock. Default 20000 |
Sweeps are resumable: re-running the same command skips finished chunks. Each chunk's results are in
<root>/<SET>/<chunk>/diff-<name>/, and <root>/SUMMARY.md has each follower's class histogram.
Today the three-way comparison (xengine adjudicate) runs only when both Forge and XMage are followers,
and it doesn't include a third engine yet. Your engine's own diff against gorge runs either way.
Draw decks from a card pool¶
xengine play -pool FILE draws both decks from a list of card names, one per line, with # comments.
Each deck is 23 different spells and 17 basics in two colours. The pool goes through the same filter as a
set: gorge must fully support each card. The pools shipped in oracle/xengine/pools/ are:
| Pool | Cards | Source |
|---|---|---|
fdn.txt |
517 | The FDN manifest |
fdn-commons.txt |
196 | FDN cards that are Pauper-legal, per MTGJSON |
Make your own with xengine pool -set <SET> [-legal <format> -mtgjson <SET>.json.gz] -out FILE, or by
hand. -pool-filter FILE leaves the named cards out, which is handy for cards your engine refuses:
fewer items then end up not_sent. The run writes pool.json with the pool's sha256, so results
record exactly which list they came from.
Report a finding¶
A diverge is a lead, not a verdict. Before anyone calls it a bug in either engine:
- Read the printed card and the Comprehensive Rules, and decide what should happen.
- Check whether other engines agree with you or with gorge.
- Rule out your driver: re-run the item from its
REPRO.mdand read your engine's own log.
If gorge is wrong, open an issue on
gorge-compliance with the item id, the repro
directory's verdict.json, and the rule. If your engine is wrong, it's yours to fix. If you do, we'd
like a link, because we keep a list
of engine bugs the harness found.
Keep the runner robust¶
- Catch panics or exceptions per row and turn them into harness rows.
- Cap each row's work. The Forge driver stops after 4,000 loop iterations without progress.
- Give a deep-recursion engine a big stack. phase.rs, for example, needs at least 16 MiB.
- If a crash can corrupt process-wide state, run rows in a worker process and restart it after a crash.