Game agents: Total War and Slay the Spire

I build the interfaces and evaluation tools that let language models play two commercial games with very different decisions, and record every decision so it can be inspected and replayed. Most of this page is about Total War, and Slay the Spire (with the small training pilots I ran there) is at the end.

The two games share one Python package for everything around the model: the client that calls it and retries replies that break the rules, spend limits, a record of the exact code each run used, and the supervisor that starts the game. Each game has its own interface, controllers, prompts, trace format and replay tools.

In Total War: WARHAMMER III the battle keeps moving while a language model decides what to do. I study that with live battle control, a small matched comparison between models, and harder scenario variants that test a tactical choice.

Open the interactive benchmark to compare scenarios, models and individual battles.

What you’re looking at

Halberds are built to stop cavalry. Here four Empire halberd units screen two units of handgunners and a cannon, which are weak in melee. The model commands six mounted Bretonnian units, and its first reply set the plan: “Advance the cavalry around the south side of the central obstruction to approach the enemy flank.”

The recorded path follows it: west along the south of the central block, then north up its western side to the guns. The gun line broke between 3:06 and 3:56, and the game declared a win at 8:07.

The records don’t show that the route decided the battle, but they do point to the knights that kept fighting the halberd line afterwards. Quoted reasons are what the model wrote in its replies, not a view of its internal reasoning.

Why latency matters

The battle runs unpaused at normal speed while the model thinks. In this battle a reply took a median 4.8 seconds, and the view behind each batch of orders was a median 5.4 seconds old by the time the game applied it. The outlines in the replay show that gap. I count the delay as part of the result.

This battle in numbers

• Setup. Cavalry assault with a stronger enemy: enemy stats raised by 10%, the game’s AI on very hard, unit size scaled to 0.75, a 20-minute limit. gpt-6-sol ran with low reasoning effort and the rules-only prompt, which explains the objectives and the interface but gives no tactical advice.
• Replies. The model was asked 84 times. 83 replies were applied (the last one, requested at 8:04, came after the battle had ended). 13 applied replies sent no new orders. Twice the harness refused a reply that broke the interface’s rules and asked again.
• Orders. 211 attack, 89 move, 22 ability and 7 behavior orders. The game rejected 10: four Mounted Yeomen Archers moves to unreachable destinations and the four shots queued behind them, one ability the unit doesn’t support, and one order to a fleeing unit.
• Timing. Age of the view behind each applied batch: median 5.4 s, mean 5.8 s, range 3.0 to 13.5 s across the 83 applied replies. Model reply time on the wall clock: median 4.8 s.
• The approach. The Grail Knights were in melee with halberds from 1:36. The Lord came within 40 m of the guns at 2:03 and was in melee from 2:13. A Knights of the Realm unit and the Questing Knights reached the guns by 2:51. At 3:06 the four knight regiments still had 162 of their 171 riders.

How it works

A Lua mod I built is the only interface to the game. It publishes structured state, showing enemy units only while the army can see them, and carries out orders. Nothing reads the screen or moves the mouse. Python controllers exchange JSON with the mod, and a model can command in two ways.

Direct control

The model orders every unit itself, one reply at a time. Between replies, units carry on with their last orders.

Model replies Orders reach units

Commander

The model sets standing tasks (engage, shoot, hold, charge, move) in the background. Scripted unit skills apply them to every fresh observation while the next plan is still being written.

Model plans Skills act

Scripted rules give reference results. Each scenario’s setup is declared up front and checked in the game’s lobby, and a rematch replays the same start. After a battle, an offline report joins the recorded movement, orders, stated reasons and outcome, and the replay at the top of this page is rebuilt from the same records.

Interface, timing and what the model can see

• The mod runs on the game’s script tick, about every 300 ms. It publishes an observation roughly every 1.2 seconds and whenever it applies a request.
• One request is in flight at a time. The next decision starts from the observation that acknowledged the last one, and each prompt states how old the previous batch was when the game applied it. Scripted controllers answer within one tick.
• The model sees its own units in full and enemy units only while its army can see them, plus the enemy roster from the lobby. Fields used for scoring and setup are removed. Values are exact numbers, so this is not the view a human player has.
• Terrain arrives as a coarse grid of blocked 32 m cells, sampled at deployment. It is a hint, not a path planner.
• “Acknowledged” means the game accepted an order, not that the order was carried out. Rejections are recorded with the game’s reason.

Controllers and reference rules

• llm orders units directly with a tactical prompt. llm_rules does the same with a rules-only prompt that covers objectives and the interface but no tactics (the matched round below uses it).
• commander keeps one plan request in flight on a worker thread. Each plan sets standing skills (engage, shoot, hold, charge, move), and the scripted skills executor applies them on every observation, about three to four decisions a second.
• Scripted references: hold, attack_nearest, rush_nearest, and skills on its own. Outside controllers plug in as a Python class.
• The report keeps reasoning effort and controller settings beside each model’s results. Mixed settings are labeled, and the matched round below uses low reasoning effort throughout.
• An offline exporter turns saved battles into training-oriented traces and has been checked on 304 saved episodes. Capturing providers’ reasoning summaries is implemented but not yet used in a paid battle.

What the evidence shows so far

The benchmark has six scenarios, each built around a different tactical decision.

In one small matched round, gpt-6-sol and gpt-6-luna each played three battles per scenario under fixed settings, against the game’s AI on very hard. In four scenarios gpt-6-sol won more battles, and in the other two both models lost all three. Three battles per model is too few to rank them, and I developed the prompts on these same scenarios.

The matched round, scenario by scenario

Fixed harness, rules-only prompt, low reasoning effort, the game’s AI on very hard, battles unpaused at normal speed. Wins out of three battles per model:

• The last reserve (siege defence): gpt-6-sol 3, gpt-6-luna 0.
• Command a swarm: gpt-6-sol 2, gpt-6-luna 0.
• Caster assault: gpt-6-sol 3, gpt-6-luna 1.
• Cavalry assault: gpt-6-sol 1, gpt-6-luna 0.
• Survive a swarm and reinforcement defence: both models lost all three.

Other rounds, and where the featured battle comes from

• Calibration, before the matched round. Six cavalry battles per controller: gpt-6-sol won 1, gpt-6-luna 0, and both scripted references 0. The battle at the top of this page is that gpt-6-sol win.
• All recorded cavalry battles together. gpt-6-sol won 2 of 9: one in calibration, one in the matched round. This is the figure under the replay. gpt-6-luna also played two other cavalry battles, one of them on medium reasoning, so its panel in the figure below pools 11.
• gpt-6.1-sol, afterwards. Three battles on each of four scenarios: it won every siege, swarm and caster battle, and one of three cavalry battles.

Making a scenario harder

More enemies don’t automatically make a better test. With two or four extra enemy Gyrocopters in the swarm battle, the model still won both times. Four extra cavalry in the siege did cause losses, but by overwhelming the breach, not by testing the reserve decision the scenario was built around. Each of these was a single battle, too few to say how hard a variant is.

The rule I took from it: a harder variant should create a choice the controller can actually make. A loss caused by missing information or an unavailable action is a different finding from a tactical failure.

In the game

The mod can also show each reply inside the game. Both captures are from early development.

Slay the Spire: full runs

In Slay the Spire a bad choice can end a run an hour later. A model plays complete Ironclad runs via CommunicationMod: at each decision it sees a summary of the public game state, the legal choices labeled one by one, its recent decisions and its own notes. When a recorded run shows a mistake, I change one part of the prompt, memory or interface and run again.

Act 1

100 HP 50

Act 2

max HP
9447

Act 3

Act 4

Floor 54 Spire Shield & Spear

Heart, not reached

• HP
• max HP
• elite
• boss
• rest
• shop

No run has won yet.

Some mistakes came from the interface rather than from the model’s decisions. With the Gambling Chip relic, the game asks at the start of a fight which cards to discard and redraw. The prompt said only “Choose up to 99 card(s) from your hand”, with the same select-and-confirm choices the model used to pick a card to upgrade, so it sometimes picked cards it meant to play. On floor 42 of one run it selected Impervious+ “to block the Transient’s full 30 damage and avoid dying this turn”, and the card went to the discard pile. The same misreading came back in later versions. The newest prompt adds one line naming what asked for the selection. Replayed over the saved traces it changes only the 385 hand-selection prompts, but it hasn’t had a full run yet.

I also ran small first-combat training pilots (paused for now), and training on full runs is planned (details below).

Interface, pilots and plans

• CommunicationMod exposes the public game state. The draw pile is shown unordered unless the Frozen Eye relic is held, and the model gets no seed, random-number state, or card and relic text.
• Runs use the Ironclad at Ascension 0 with gpt-6-luna on low reasoning effort. The prompt, memory and interface are on their tenth version, each changing one thing, and the furthest run was on version 8.
• The first-combat pilots compared a scripted heuristic, PPO and a LoRA fine-tune of Qwen3-0.6B, run locally on fixed floor-1 fights. The LoRA matched the heuristic’s average remaining HP on a three-encounter development set (76.33 against 76.00, over 12 attempts), but PPO changed its weights without improving play.
• Planned, with only a code skeleton so far: reinforcement learning on full runs with open-weight models on remote hardware, starting with Qwen3.6-35B-A3B and then GLM-5.3.