Engine integration notes¶
How each engine is wired into the harness: where its driver lives, which of the engine's own extension points the driver uses, what it pins, and what had to change in the engine (if anything). Forge and XMage are the two working followers, so their sections double as worked examples for a new engine. The phase.rs and ManaBrew sections are plans, written from reading and building their code in November 2026; neither has replayed a scenario yet.
If you work on one of these engines and something here is wrong, or you'd like to own the follower yourselves, please open an issue.
| Engine | Where the driver lives | Engine changes | Pins |
|---|---|---|---|
| Forge | forge-oracle/ in our Forge fork |
None for testing; the fork also carries card-script fixes and card support | FORGE_ORACLE_REF (driver), gorge's FORGE_REF (scripts) |
| XMage | oracle/xmageoracle/ in gorge-compliance |
None: upstream XMage, unmodified | XMAGE_REF |
| phase.rs | planned: followers/phase/ in gorge-compliance |
None planned | planned: PHASE_PIN (engine commit + card-data hash) |
| ManaBrew | planned: a crate in a ManaBrew fork | Three small hooks, meant for upstream | planned: ManaBrew commit + gorge's FORGE_REF |
Who leads¶
In every comparison, gorge leads and the other engine follows. gorge plays the game, with its own exploring bot making every choice, and records each decision. The follower replays the position and those decisions; it never chooses anything itself. A prompt with no recorded answer is a strict miss, not an invitation to improvise.
That shapes what the harness can and can't see:
- It sees every place where the follower, given the same choices, reaches a different state, and
every prompt one engine poses and the other doesn't (
decision_shape). - It can't see a line of play gorge's bot never takes, or a legal action gorge fails to offer: the follower is never asked to try it. Coverage is the positions gorge reaches.
- The follower's AI never plays. XMage's and Forge's AIs only fill in when a driver runs loose, which new followers shouldn't do.
The one exception is dirac, which is compared in lockstep rather than through this
protocol. There gorge also leads by default, but kfuzz -lead kernel lets dirac choose and gorge follow.
Forge¶
Driver. forge-oracle/ in our fork, about 4,800 lines of Java for scenario replay (plus a bridge
that lets Forge's AI play whole matches against a gorge bot, which this harness doesn't use). It's GPL,
like Forge, so it and its output stay out of the Apache-2.0 repositories. scripts/forge-oracle-setup.sh
builds it at FORGE_ORACLE_REF, and scripts/forge-oracle-run.sh is the runner.
Forge's own extension points it uses. No engine change was needed for testing:
- Decisions.
ScriptedControllerextends Forge'sPlayerControllerAi, so every question Forge asks a player (targets, modes, costs, blockers, ordering) arrives as a method call the driver answers fromgorge_decisions.ScriptedCostDecisiondoes the same for cost payment. - Placing a board.
SetupBuilderadds cards straight to each player's zones, while Forge's trigger handler hasChangesZone,ChangesZoneAllandCounterAddedtriggers suppressed (suppressMode), so nothing enters with an ETB trigger. It then runs Forge's own end-of-turn cleanup to clear the placement's "this turn" history, and starts the game withsetupFirstTurn. - Randomness.
MyRandom.setRandom, seeded from a CRC32 of the item id, so reruns are identical. - Driving.
StepMachineadvances Forge's phase handler and takes a snapshot at each checkpoint.
What the fork carries besides the driver.
- Card-script fixes. Where a Forge script disagrees with the printed card, we fix it on the fork as well as reporting it upstream, so the comparison isn't measuring a known script bug.
- Card support for new cards gorge needed, with the small engine additions that came with them.
- The pin rule. gorge compiles Forge's scripts at its
FORGE_REF. The runner refuses to start unlessFORGE_REFis an ancestor of the driver's commit and thecardsfolderandtokenscriptstrees are byte-identical at both (forge-oracle-setup.sh --check-pin). Otherwise a script difference would look like an engine disagreement.
Known quirks, recorded in the registry so the classifier doesn't call them bugs: Forge skips a combat
damage step when no creature is left in combat (SkipsDamageStep), gives priority in cleanup after an
end-the-turn effect (StopsInCleanup), poses trailing cleanup discards (TrailingMisses), acts on only
some cast_mode values (CastModes), and stops a tape at the scenario turn plus four (TurnCapMsg).
XMage¶
Driver. oracle/xmageoracle/src/org/mage/test/oracle/ScenarioReplay.java in gorge-compliance, with
its own tests beside it. There is no XMage fork: scripts/xmage-oracle-setup.sh clones upstream
magefree/mage at XMAGE_REF, builds its Mage.Tests module, and
compiles the driver against it. scripts/xmage-oracle-run.sh is the runner.
XMage's own extension points it uses. XMage's unit-test framework is the seam:
- Placing a board and acting.
ScenarioReplayextendsCardTestPlayerBase, the base class of XMage's own card tests. ItsaddCardplaces cards directly into zones, andcastSpell,activateAbility,addTarget,setChoiceandsetModeChoicescript the actions and answers. - Decisions.
ScriptedChoicePlayerextends the framework'sTestPlayerand answers the prompts the framework's scripting doesn't reach. Strict choose mode makes an unanswered prompt an error instead of an AI guess. - Randomness.
RandomUtil.setSeed. - Card data. XMage's own card database, copied per run.
What the harness adds for XMage. xengine xprep rewrites each item for XMage before it's sent:
XMage's card spellings, the answer order its prompts expect, and refusals for cards XMage doesn't have
(xmage_lacks_card). Whole-game tapes are off by default (XENGINE_XMAGE_TAPES=1 turns them on).
Known quirks: XMage taps mana and casts within one priority out of turn (TapsWithCast), matches a
scripted trigger order by text or source name (TriggerOrderByName), and reports a missing card as
Couldn't find a card: (LacksCardMsg).
phase.rs¶
phase.rs parses Oracle text from MTGJSON into its own card data. It is the only engine here that doesn't share an encoding with gorge, Forge or XMage, so it is the follower we expect to add the most independent evidence.
What we read. Commit c53dd8a (v0.122.0). Its release build of phase-engine compiles cleanly on
the pinned toolchain (nightly-2026-04-19, from rust-toolchain.toml). From scratch that took 12
minutes on our machine.
What fits well¶
- A debug API that places cards raw.
create_debug_cardswithrun_etb: false, plus debug actions for tapped state, counters, summoning sickness, attachments, life, tokens and face state, covers the setup contract without engine changes. The test helperadd_real_cardruns the real entry pipeline, so enters-tapped and enters-with-counters apply; a follower must not use it for setup. - One entry point for actions. Every action goes through
apply(state, actor, GameAction), and the engine lists the legal candidates for each prompt (AiDecisionContract::issue). A scripted answer can be checked against that list before it's submitted, which is exactly what a strict driver needs. - Stable identity. An
ObjectIdsurvives zone changes, andincarnationmarks a new object under CR 400.7. The follower's ref table binds gorge refs to(ObjectId, incarnation). - An honest support flag. Each face has parser gaps (
card_face_gaps) and a list of known misparses (confirmed_misparse). The follower's preflight refuses a card if any face has either.
Card support¶
From phase's published coverage data (card-data hash d0ffa159dcde97c7, engine b14f974, MTGJSON
5.3.0+20261110):
| Pool | Supported |
|---|---|
| FDN | 494 of 517 (95.6%) |
| FDN commons (Pauper-legal) | 196 of 196 |
| Pauper-legal cards | 10,405 of 11,123 (93.5%) |
| Standard | 5,129 of 5,725 (89.6%) |
21 of FDN's 23 unsupported cards parse without a gap but carry a known misparse, so a preflight that checks only parser gaps would let them through. "Supported" means no known gap; the follower is how we'd learn about the unknown ones.
Hurdles¶
- Matching abilities. phase indexes abilities by its own parse of the Oracle text, and gorge by the
Forge script. The follower matches on
neutral.ability: class, then ordinal among abilities of that class, then the text as a check. - Names. phase names tokens by CR 111.4 ("Goblin"), and Forge's scripts by "Goblin Token". The
comparator anonymises tokens, so this only matters for a token's
namefield. Split, Room and multi-face cards, and accented names, need mapping to gorge's spellings. - Prompts gorge may not record. phase asks some choices that gorge and Forge resolve internally:
replacement-effect order, combat damage order among several blockers, Phyrexian payment per pip, and
the bottom order after scry. Until gorge records them, they become strict misses (
decision_shape). We'll add recording where the counts justify it. - phase-only conveniences. The batch-resolve prompts (
ResolveAll*) are never started by a follower, and cards with real-timePhysicalContestprompts are refused. - Speed (CR 702.179). We haven't found a setter for a player's speed, so items that set it will be harness rows for now.
- Stack size. The engine needs at least a 16 MiB stack (
RUST_MIN_STACK). The follower runs it on a thread with 64 MiB and catches panics per row. - Pace of change. phase made 15 releases between 2026-10-12 and 2026-11-10, and its
GameActionandWaitingForenums grow often. The follower pins one engine commit together with the card data built from it, and matches those enums exhaustively, so a new variant is a compile error rather than a silent gap. The card-data metadata named commitb14f974while the repository's head wasc53dd8a, so the pin has to name both.
Plan¶
The follower lives in gorge-compliance under followers/phase/. It depends only on phase-engine's
public API, so it could move into phase's workspace unchanged if you want it. It carries no card fixes:
its point is to report phase as it is.
| Milestone | Scope | Done when |
|---|---|---|
| P0 | Toolchain, pin, preflight | The preflight reproduces the numbers above |
| P1 | Setup only | The "setup" snapshot agrees on at least 95% of FDN-commons scenarios |
| P2 | Cast, activate, resolve, pass; targets, modes, X, yes/no | Every FDN row agrees or has an explained class |
| P3 | Windows: combat, pass_to, attach |
decision_shape under 20% of sent windows |
| P4 | Whole-game tapes | Tapes run to the lowering's cut |
phase's contributor guide (docs/AI-CONTRIBUTOR.md) is a procedure for single-card fixes, with
strict rules for outside pull requests. A follower is a different kind of contribution, so we'll start
with an issue rather than a PR. Related prior work:
kevinbrar/mtg-rl-sim drives phase from Python for
reinforcement learning, and solved several of the same mapping problems.
ManaBrew¶
ManaBrew is a Rust port of Forge's engine that reads Forge's card scripts. It is in the Forge family, so it doesn't add an independent reading of the cards. What it adds is a second implementation of Forge's rules engine over the same scripts: a check on Forge, on our Forge driver, and on ManaBrew itself. (gorge also serves the ManaBrew wire protocol for ManaBrew clients; that is unrelated to the follower.)
What we read. Commit dadee73. manabrew-engine builds on stable Rust 1.98 in under a minute.
The card scripts come from its Forge submodule, witchesofthehill/forge@be9e0c8.
What fits well¶
- The same card scripts as gorge. The
abilitiesscript lines and gorge's mode labels match ManaBrew's parsed scripts directly, so a ManaBrew follower needs noneutralmatching. - A headless game is already possible. The (now deprecated) parity runner builds a game without the starting-player roll or mulligans, places cards and runs turns, which is the shape a follower needs.
- Card data from any Forge tree. Scripts are read from an archive built from a
res/directory, so the follower can build it from gorge's pinned Forge scripts (FORGE_REF) rather than ManaBrew's own submodule. Only 4 FDN scripts differ between the two, which makes FDN a safe place to start.
Card support¶
ManaBrew doesn't mark cards as unsupported, so we scanned the scripts statically for vocabulary the Rust code doesn't know: effect types, trigger modes, static and replacement modes, and parameter keys.
| Pool | No red flag |
|---|---|
| FDN | 479 of 517 (92.6%) |
| All of Forge's cardsfolder | 30,138 of 34,442 (87.5%) |
The flagged FDN cards mostly use a parameter key that appears nowhere in the Rust source. Across the
whole cardsfolder, the unknown effect types are FlipCoin (84 cards), PutSticker (38), Empower
(35), Recruit (10), Assimilate (8) and HealDamage (3).
FlipCoin is worth a look on its own. Forge renamed the effect from FlipACoin on 2026-04-02, and
ManaBrew's own Forge submodule has the new name, but the Rust code only knows the old one. So all 84
coin-flip cards currently do nothing in ManaBrew itself, not just in our follower.
A static scan only shows that the vocabulary exists, not that the behaviour is right.
Hurdles¶
- Silent fallbacks. Where ManaBrew meets something it doesn't know, it carries on: an unknown effect
does nothing, an unknown trigger mode drops the trigger, an unknown static or replacement is inert, an
unsupported trigger condition counts as true, and an unknown count is 0. 74 of the 83
PlayerAgentmethods have defaults that pick the first, minimum, false or true answer. In a follower, each of those would look like agreement or like a rules difference. So the follower:- refuses any card with flagged vocabulary in the preflight;
- implements all 83 agent methods, failing on any prompt it can't answer from the script;
- fails a row on any engine diagnostic, on an illegal action the engine downgraded to a pass, and on any random draw it didn't script.
- Engine hooks. Three small engine changes, each useful beyond gorge:
- a hook that lets the agent change state from inside the priority loop (to add pool mana and to
apply
tapped_after_untap), like the existing restore-request hook; - a way to place a card in a zone with given flags and no entry pipeline, since
move_cardapplies entry replacements and adds loyalty; - a sink for the engine's diagnostic messages ("Unimplemented effect API type" and the like), which
today go to stderr. It would have caught the
FlipCoinrename.
- a hook that lets the agent change state from inside the priority loop (to add pool mana and to
apply
- Trigger order. The engine orders simultaneous triggers itself, in APNAP order. Where gorge chose an order, the item can't be matched until there is a hook for it.
- Licence. ManaBrew is AGPL. The follower lives in a fork and talks to the harness through files, like Forge's GPL driver today. Its output is never committed.
Plan¶
| Milestone | Scope | Done when |
|---|---|---|
| M0 | A probe: build, card archive from FORGE_REF, place a board and cast one spell |
Open questions answered (back-face setup, combat flags, cost callbacks) |
| M1 | Hooks 2 and 3, setup, snapshots, preflight | "setup" agrees on at least 95% of FDN-commons items that pass the preflight |
| M2 | Hook 1, cast, activate, resolve, the scripted agent | FDN rows agree or have an explained class; Forge-vs-ManaBrew differences triaged |
| M3 | Combat, pass_to, windows |
A commons sweep beside Forge and XMage |
| M4 | Whole-game tapes | Tapes run past Forge's turn cap |
ManaBrew comes after phase.rs in our order of work, because phase.rs adds a new family. We'd like to offer the three hooks upstream as generic agent and diagnostic hooks, following ManaBrew's contribution rules. Since ManaBrew's parity harness is deprecated, a nightly gorge-vs-ManaBrew run could also take over its job.