Your game AI has a tell: finding and fixing the always-9 exploit
Companion to “How the bot reads your hand: belief-MCTS in the Black and White AI.” That post explained how the 관심법 bot reasons about your hidden tiles. This one is about a bug in how it plays — a deterministic tell that a human could farm — and the surprisingly deep rabbit hole of fixing it.
The tell
Quick game recap: two players hold tiles 1–9; each round both reveal one in secret; higher wins (but 1 beats 9); first to 5 round-wins takes the game. Color is public (○ odd / ● even), number hidden.
Here’s the bug. When the human leads with any black (even) tile, the bot — as the second player — almost always answered with the 9:
policy vs a black opener: 9: 99.9% everything else: ~0%
Playing 9 does win the round — 9 beats every even tile. So at a glance it looks fine. But look closer: to beat a 2, the bot just spent its strongest card. And it does this every single time. That’s not a strategy; it’s a tell.
Why a tell is a bug (even when each move “wins”)
A human notices the pattern and starts opening with their lowest black tile every time they lead. The bot dutifully crushes that 2… with its 9. The human loses the round but trades a worthless 2 for the bot’s best card. Repeat, and the human bleeds the bot’s strong cards dry, then wins the rounds that matter.
We didn’t want to guess whether this mattered, so we built the exploiter and measured it. A NineBaiter that always opens its lowest black tile, seated first every game:
bot winrate vs NineBaiter
deployed bot ████░░░░░░░░░░░░░░ 24%
The bot won 24% of games. A human who finds this one trick beats it three games out of four. The tell was a genuine, severe hole.
Two ways a policy collapses
The fix has to undo a collapse — the policy putting ~100% on one move — in two different places.
1. The policy itself collapsed. The network is trained to imitate strong play, and the training signal rewarded the move that won the round. Against a black opener, the 9 is the only move that wins for certain, so the gradient piled all the probability onto it. The network never learned that 7 and 8 (which win 75% of the time and keep the 9) are perfectly reasonable answers too.
2. The search re-collapsed it. The strong bot doesn’t just play the network’s hunch — it runs a search (belief-MCTS) on top. Search is supposed to improve the policy. But standard game-tree search concentrates its effort on the single highest-value move — so even after we taught the network to spread its probability over {7, 8, 9}, the search would dutifully re-collapse it back onto 9.
The deeper lesson, which is true of every hidden-information game: a deterministic strategy is exploitable by definition. If the opponent can predict you, they can plan against you. The unbeatable strategies in these games are mixed — you randomize among good options precisely so you can’t be read. The bot’s problem wasn’t that 9 is wrong; it’s that always 9 is readable.
The fix, part 1: teach the model to mix (not the dice)
The tempting fix is play-time randomness — just sample the move with some “temperature.” We deliberately didn’t. Temperature on a 99.9% spike is blunt noise: crank it high enough to shake loose the 9 and you also start playing the losing 2s and 3s.
Instead we fixed the training target. When the search records what it learned, we blend its target with a small dose of “which tiles can win this round, and by how much” — spread over the good moves only ({7, 8, 9} get probability; the losers stay at zero) — and we weight this blend toward the early rounds, where uncertainty is highest. The model now learns a target that looks like a real distribution over reasonable answers:
black-opener policy, before → after
before: 9: 99.9% (entropy 0.012 bits)
after: 9: 65% 7: 11% 8: 9% ... (entropy 1.85 bits)
The “entropy” number is just how spread out the choice is. It went from a near-certainty (0.012) to a genuine mix (1.85) — over the right moves.
The fix, part 2: a search that respects the mix
Now the subtle one. We re-ran the search with the new, spread-out policy as its guide… and the bot still played 9 almost every time. The search was re-collapsing the mix.
The culprit turned out to be a single interaction. To add variety, the bot samples its final move with a temperature below 1 — which squares the distribution. Squaring a mix like 9:71% 7:9% 8:8% turns it right back into 9:96%. The variety knob was undoing the very mixing we’d just trained in.
The fix: let the deployed search blend its move choice with the network’s (now-mixed) learned prior, weighted toward early rounds, and sample that blend linearly — no squaring. This isn’t dice; it’s the search deferring to what the model learned about when to mix. With it on:
black-opener: share of "9"
search, before ████████████████░ 97%
search, after ████████████░░░░░ 73%
73% on the 9, 27% on a cheaper winner — enough that the bait stops paying.
Did it work? (and the honest catch)
bot winrate vs NineBaiter
before ████░░░░░░░░░░░░░░ 24%
after █████████░░░░░░░░░ 54%
Bait-resistance more than doubled, 24% → 54%. And both bot personas benefit: the search-based bot (관심법) goes to 54%, and the faster policy-only bot (중꺽마), which plays its top-two picks, now finds a cheaper winner alongside the 9 and lands at 57% against the same baiter. The exploit is closed on the live bot.
But here’s the honest part. The old bot crushed certain predictable bot-opponents — it would win 92% against one that always plays the textbook move, because it learned to exploit their predictability. The new, mixier bot wins only ~50% against that same opponent. We traded exploiting for being less exploitable.
That’s not a regression; it’s the whole point. Those crushed opponents were deterministic bots, not humans. Against actual human-style play the bot is still strong (60% vs random play, 77% vs a card-hoarder) — it just can’t be farmed by a one-trick bait anymore. Moving from “spiky — crushes the predictable, folds to the bait” toward “flat — solid against everything” is exactly the direction of a game-theoretic equilibrium.
The real destination
Our fix is a good, shippable approximation — but it’s still a hand-tuned amount of mixing. The principled endpoint is to compute the game’s Nash equilibrium (with a method like Counterfactual Regret Minimization), which finds the exactly right mix at every position: unexploitable by construction, and — beautifully — it mixes more early (where the opponent’s hand is wide open) and sharpens late (where the line is forced), all on its own, with no knob to turn.
It’s a bigger build, though. We counted the game’s distinct decision situations (“information sets”) and found well over ten million of them — too many to just tabulate. So the equilibrium needs a sampling method (Monte-Carlo CFR) or a neural approximation (Deep CFR), or solving just the early game where this exploit lives. That’s the next chapter.
For now: the bot on board-game.wintersalmon.com no longer always cashes its 9. Open black low a few times and see for yourself.
The numbers here are from real measurements on the deployed model; the exact percentages move a little run to run, but the story — a deterministic tell, a baiter that farmed it, and a model taught to mix over the right moves — is exactly what happened.
