The AI took two moves when one would win — a YINSH value saturation story

The AI had a one-move win and didn’t take it

At score 2:2 in a YINSH match against the staging AI, the bot had a single move that would end the game on the spot — slide one ring, flip a piece, complete a row of five. It didn’t take it. It played a slower setup move instead, handed me a free turn, then won two moves later. Final score: 3:2. It won, but it dawdled.

YINSH is one of the board games in this LLM-driven build experiment. Every multiplayer game in the project is written permanently to MongoDB, so nothing disappears when a game ends and every move can be replayed. That meant I could return to the exact position and ask the AI why it hesitated.

TL;DR

• At 2:2, the AI had exactly one instant-win move: g6→i6. It played d5→d2 instead (staging room 6a434b18093697526ddb151f, bot ai-fp-gen208, 2026-06-30).
• Running the AI on that exact position produced a confidence score of 0.98 out of 1.0 for both moves — it genuinely could not tell them apart.
• The cause: the AI was trained only on who wins, never on when. A win in 2 moves and a win in 20 moves both score “+1”.
• More training makes this worse. A stronger, more certain AI saturates even harder and grows more indifferent to urgency.
• The fix is a “proven-win” shortcut in the look-ahead search — a focused change in packages/yinsh-ai-core/src/engines/mcts-engine.ts, no retraining, 115/115 tests pass.

The forensics: every game is permanent in the database

All multiplayer games in this project are fully persisted to MongoDB in two collections: game_rooms stores the final board state, and game_room_events stores every move in order by sequenceNumber. There is no expiry — completed games stay forever. The ws-relay (the live game server) holds no state of its own; it reads MongoDB change streams, so a disconnection or game-over loses nothing.

The specific game — room 6a434b18093697526ddb151f, completed 2026-06-30 at 14:05 KST, bot ai-fp-gen208 — had 75 events. I exported the command log and replayed it through @cloudnest/yinsh-engine to reconstruct every intermediate board position. The scoring timeline is clear: after four ring removals (two per side) the score reaches 2:2 at move 70. The AI moves at sequence 71.

Of the AI’s 25 legal moves at that point, exactly one wins immediately: g6→i6. It is not subtle — the ring slides over an opponent’s piece at h6, flips it, and completes the five-in-a-row h3–h4–h5–h6–h7. The AI played d5→d2 instead, which scores nothing and leaves the game continuing.

The AI cannot tell “win now” from “win later”

I ran the real AI model — gen-205, the same 128×10 architecture as the deployed gen-208 — on the exact sequence-71 position with 400 look-aheads and no randomness:

• Raw confidence score for the position: 0.98 / 1.0 — already deeply certain it is winning.
• Look-ahead visit split: d5 received 226 visits (57%), g6 received 138 (35%). The winning ring was the second-most-explored option.
• Final move selected: d5 — the search reproduced the exact “wrong” move from the real game.
• Forced to lift g6: it immediately plays g6→i6 and finds the win. The instant-win move was not hidden from search; it just lost the vote.

This is value saturation — a known behavior in AlphaZero-style training. Because the AI learned only from raw outcomes (+1 for win, −1 for loss, 0 for draw), a position where victory is two moves away and a position where victory is twenty moves away both score approximately +1. The confidence meter maxes out at the top and can no longer rank options by urgency.

The analogy: a student told only “pass the exam,” never “finish early.” Once passing is assured, any strategy that leads to a pass is equally good — there is no reason to rush.

What makes this harder to fix than it sounds: more training makes it worse. A stronger, more certain AI pushes its scores even closer to exactly 1.0 for all winning positions, becoming even more indifferent. The gen-205 champion — the best-trained version available — makes the identical mistake in testing. This is not under-training; it is a structural gap in what the AI was asked to care about.

Proven win: if you can prove you win right now, take it

The fix is a rule added to the look-ahead search itself — no retraining, no new data, no waiting for the next GPU campaign. In packages/yinsh-ai-core/src/engines/mcts-engine.ts, the change implements what is called an MCTS-Solver: while exploring possible futures, if any move leads to a guaranteed, no-escape win for the current player, route the entire remaining look-ahead budget into that move immediately, and record the proof so it can cascade upward through parent moves.

YINSH’s winning turn is a four-step chain (place marker, move ring, select the five-in-a-row, remove ring). The fix handles this by propagating proven wins up through each step, correctly flipping the “whose win” label at each player transition. When no terminal is reached within the search budget, nothing changes — behavior is identical to before.

The two effects are linked: the deployed AI now plays the instant win when it has one, and the self-play training loop also applies the same rule. That second effect matters because each position’s look-ahead visit distribution is the policy target — what the neural network learns to imitate. Before the fix, the training data pointed the policy toward d5 (56% of visits) and away from g6 (35%), actively teaching future generations to dawdle. After the fix, the visit mass concentrates on the forced win.

A regression test (src/__tests__/mcts-terminal-win.test.ts) loads the exact sequence-71 position from a saved fixture (fixtures/seq71-mate-position.json) and asserts the engine plays PLACE_MARKER g6 → MOVE_RING g6→i6. The test uses a value-blind network — all outputs set to zero — deliberately, proving the fix comes from the search rule and not from a network that happens to know the answer. Full suite: 115/115 pass, typecheck clean. Confirmed with the real model: gen-205.onnx now plays g6→i6 where it previously played d5.

Bots stop toying with you once they’ve won

The immediate consequence is practical: an AI with a winning move now takes it. The longer-term effect is in training quality — self-play games end decisively on the winning move, generating sharper data for future generations and shortening average game lengths. The deeper fix (a discount factor applied to the value target, teaching the network to prefer early wins in its weights) remains possible but is not required for this.

Keywords

• YINSH — an abstract strategy board game where you win by removing three of your rings; you remove a ring by forming a row of five markers of your color.
• MCTS (Monte Carlo Tree Search) — a move-selection method that simulates many possible game continuations, favoring moves that tend to lead to wins; more simulations mean a better-informed choice.
• Value saturation — when a neural network’s confidence score maxes out (+1 for certain win) across many different positions, losing the ability to rank them by how urgent a win is.
• AlphaZero-style training — a self-play training method where outcomes are scored as +1 (win), −1 (loss), 0 (draw), with no bonus for speed; the approach popularized by DeepMind’s AlphaZero.
• MCTS-Solver — an extension to MCTS that recognizes mathematically proven wins and losses during search, routing all remaining look-ahead budget toward the forced win.
• Policy target — the distribution of look-ahead visits at a position; this is what the neural network learns to imitate in training, so it directly shapes future AI behavior.
• Discount factor — a multiplier less than 1 applied per move to the value target, making earlier wins score higher than later ones; the “full cure” for value saturation that requires retraining.
• MongoDB — the database where every game move is stored permanently; nothing is lost when a game ends or a server restarts.

References

• docs/task-log/20260630-yinsh-missed-mate-diagnosis/01-diagnosis.md — the full investigation: data provenance, move reconstruction table, engine probe results, and fix options with pros/cons.
• packages/yinsh-ai-core/src/engines/mcts-engine.ts — where the MCTS-Solver terminal-win shortcut (MctsNode.solved, edgeProvenValue, updateSolved) was implemented.
• packages/yinsh-ai-core/src/__tests__/mcts-terminal-win.test.ts — the regression test that loads the sequence-71 position and asserts the engine plays g6→i6 with a value-blind network.
• packages/yinsh-ai-core/src/__tests__/fixtures/seq71-mate-position.json — the board state exported from the real game, used as the test fixture.
• Room 6a434b18093697526ddb151f — the MongoDB staging game record (multiplayer-games db, game_rooms + game_room_events collections), 75 events, ai-fp-gen208 vs human, final 3:2.
• YINSH rules (GIPF project) — the official rules, including how rings are removed and what constitutes a win.
• MCTS-Solver (Winands et al., 2008) — the paper introducing proven-win propagation inside MCTS.

AI workflow note

The investigation used Claude Code to replay the move log and probe the live engine — compressing what could have been a multi-day manual debug into one session. The prompt pattern that worked was a strict measure-first discipline: establish the forensic record first (export the game, replay moves, enumerate legal moves at the critical position), then run the engine on that exact board before drawing any conclusions. The agent’s first instinct was to frame this as a training problem requiring retraining; the correct move was to run the engine probe first, which showed the winning move was already visible to search (138 of 400 look-aheads) and definitively ruled out the more expensive fix path. The MCTS-Solver change and regression test came only after the diagnosis was solid, not before.

#ai #yinsh #debugging