Neon Protocol v0.2: I let a panel of AIs playtest my game. Here’s what they found

Version 0.1 of Neon Protocol: Uprising went out with a lot of content and no outside players. For v0.2 I wanted to fix the first hour before adding anything new, and to do that from evidence rather than guesswork. So the “outside players” were AIs, each given a different personality.

The panel

There were two kinds of tester. Two language-model players played the game cold in a browser and wrote up their experience. Nao has never played a tactics game, skims long text and quits when confused. Kenji has finished every Advance Wars and XCOM, reads every tooltip and checks the numbers.

Alongside them, six bot personalities took the human side of the campaign and logged every game: a first-timer, a distracted phone player, a rusher, a turtle, an expander and a min-maxer. Each one bends the game’s own AI in a different way, from how many units it bothers to move each turn to how sloppy its target choice is. Together they played more than 150 games.

What they found

  • A crash on the very first screen. Nao opened the game in a small window and got a black battle map. One bad frame stopped the game drawing for good. v0.2 fixes the cause and makes sure a single bad frame can never freeze the game again.
  • An enemy that snowballed. The bots kept losing early missions, so I watched one. By day 41 the Overmind had 40 units to the player’s 2, built entirely on captured income. Enemy armies in the campaign are now capped (14 on Recruit, 24 on Veteran).
  • A trap. Kenji found that the Fire menu would happily offer a civilian or a petrol station as your only target, and shooting either costs you rank. Civilians are no longer offered as targets.
  • A preview that left things out. The damage preview was accurate every time Kenji checked it, but it ignored effects applied after the dice roll, and gave no reason for differences. It now includes Rend, Aegis, shields and splash, plus a line explaining the factors.
  • The story arrived too late. Nao skipped the first line of dialogue by accident and spent the first mission not knowing who the Overmind was. There is now a short prologue, and a click on a title card can no longer skip dialogue.

What changed

New players now start on Recruit, which is a real first-timer mode: fewer enemies, more money and a weaker enemy economy. The hardest early missions were rebalanced, and in those missions the first-timer bot’s losses fell from five in six to about one in six. The first click on a target previews the damage and the second fires. Faint red tiles show where a unit could shoot from. After every enemy turn a “While you waited” panel lists what happened, with links that jump to each event. There is a glossary for the jargon and a “Report a problem” button in the pause menu. The full list is on the game page and in the in-game “What’s new” screen.

What the AIs can’t tell me

The bots are only as good as the game’s own AI, so many of their games end in stalemate where a person would push through. One mission, 1-6 “Dead Air”, still beats every bot personality, and I have left it flagged for v0.3 rather than keep guessing. The language-model players are better at saying what confused them than at playing well. None of this replaces real people, which is where you come in: if you play it and something feels unfair or broken, the pause menu will write the bug report for you.