Spending 10 months building the most human-like chess bot

Everyone knows playing against an engine like Stockfish is nothing like playing a human. The existing human-like models (Allie 2.0, Maia-3) weren’t quite right for my use case, and I became a bit obsessed with making the best one.

I wanted a sparring partner that plays just like a random opponent on Lichess, at any rating and any time control. So over the last ten months I built one. It’s called Chessbook-1, and it’s the bot behind the training games on Chessbook.

The best models available right now for this are Allie 2.0 and Maia-3. Maia-3 doesn’t match playing strength well, plays too strongly at low ratings, isn’t trained on low-time situations or non-blitz time controls, and it doesn’t predict how long a human will take to play a move.

Allie 2.0 is by a good margin the best predictor overall, but predicts time usage independently of the move, and its strength drifts at low-Elo and high-Elo play.

So I made my own, and here’s how it compares to Allie 2.0 and Maia-3.

TL;DR: Chessbook-1 predicts human moves well, is the only model calibrated to play at the right strength at every rating, and it errs in time scrambles like a human would. It predicts time usage well, and is aware of the time it uses.

How we measured it

Two sets of positions, both from Lichess games played in September 2026, a month after every model’s training data ends (Chessbook-1 trained on games through July 2026, Maia-3 through July 2025, Allie 2.0 through August 2026):

• Any game at all. 30,000 positions drawn at random from every rated game that month: every time control from ultrabullet to classical, every rating, no clock floor. A third of them are bullet.
• Blitz, Maia-3’s home field. 28,000 positions from rated blitz games, 4,000 per rating band from 600 to 2600+. Maia-3 was trained on blitz only, so this is the comparison on its own terms.

Every legal move in every position was scored by Stockfish 18. Each model gets the position, the players’ ratings and any other features it accepts (Chessbook-1 and Allie read the clocks and the move history; Maia-3 reads the history but not the clock), and outputs a probability for every legal move. Then we compare that distribution with what the human played.

Does it predict what a human would play?

The easiest measure is top-1 accuracy: how often the model’s top prediction is the move the human actually played. The other number worth watching is how much probability the model put on the move the human chose, averaged over positions: a model that spreads its guesses sensibly scores well here even when its favourite move is wrong.

Start with a random position from a random game: any time control, any clock, any rating.

Allie 2.0, at 28 times the size and reading the whole game, is about a point ahead of both in every band but the lowest and the highest. Between Chessbook-1 and Maia-3, the two are level below 1800 and Chessbook-1 pulls ahead from 1800 up; pooled over all 30,000 positions it leads Maia-3 on both measures by a small but real margin, and the gap is widest in bullet games and in the last seconds on the clock, where Maia-3’s training data runs out.

This isn’t really a fair comparison with Maia-3 though. That model is trained only on blitz games, and every position after a player’s clock first drops under 30 seconds is thrown out of its training data, so a random Lichess position is often out-of-distribution. So how does Chessbook-1 do on Maia-3’s own terms: blitz games, with the time scrambles filtered out?

A three-way statistical tie. Over all 28,000 positions Allie 2.0 gets 57.5% top-1, Maia-3 57.3% and Chessbook-1 57.3% (27.9%, 27.6% and 27.3% probability on the human’s move); the paired differences are within ±0.3 points. Band by band everything is within noise except 600–1000, where Maia-3 and Allie are about a point ahead on top-1 (the per-band confidence intervals are ±0.8 to ±1.0 points). For scale, Maia-2 is at 50% pooled, and a Stockfish limited to the player’s rating agrees with the human only 24–40% of the time, which is the whole problem with engine-based bots.

So our model matches Maia-3 on its own strengths, and beats it in the areas it’s not quite trained for. It falls short of Allie 2.0 on raw prediction, which is expected given the 28x size difference in the models, and it makes up for it by matching strength better and with better time usage.

Does it play at the right strength?

A move predictor is not automatically a good opponent. To give an extreme example, if you put 95% probability on the right move every time, but put 5% probability on weird blunders, then you’ll never play a competitive game against an opponent who knows what they’re doing, because you’ll be blundering twice in a 40-move game and it only takes one blunder to lose the game. Maia-3 gets around this a bit by not sampling from its distribution, taking the top move instead. But this causes it to over-perform at low ratings, and it still struggles at high ratings. Chessbook-1 can sample from its distribution at every rating, and play almost exactly at that level, until 2600+.

To measure playing strength, we take positions where the evaluation is within two pawns of equal. Here is each bot’s average centipawn loss relative to the humans who actually played those positions, band by band. Zero means it loses exactly as much as they do; below zero means it plays stronger than they do:

And the same for the blunder rate, which is maybe a better representation because it’s hard to feel a 10 centipawn loss average difference. But 2 blunders per game vs 3 feels very different over time:

Chessbook-1 tracks the humans in every band on both measures; its bars barely leave zero. Maia-3 and Allie 2.0 match the humans up to about 2200 and then drift too weak as the rating climbs, until at 2600+ both blunder nearly twice as often as the players they are imitating. Allie also runs noticeably too strong at the bottom of the scale. (A rating-limited Stockfish goes the other way: it blunders about half as often as a human of the same rating, which is why it feels like playing a strong player who occasionally hangs a piece.)

Another way to see the same thing: take the model’s probability mass on moves that are mistakes and compare it with how often humans at that rating actually make mistakes. A perfectly calibrated opponent errs exactly as often as they do.

Chessbook-1 stays within 9% of the humans in every band, 18% at 2600+. Maia-3 and Allie 2.0 are fine up to 1800 and then come apart: 63% and 75% more mistakes than a 2600+ player. The stronger the player you imitate, the more these models over-represent the rare mistakes, and nothing in those models corrects for it.

Time scrambles

Humans play differently with limited time remaining on the clock, and a bot that does not know the clock won’t play as if it’s under any pressure at all. Maia-3 has no clock input. Chessbook-1 trains without any filter for clock, so it sees both players’ clocks, the base time and the increment.

On the 2,600 positions of the random set where the player had under 20 seconds left:

Pooled over all bands Chessbook-1 predicts the human’s move 54.8% of the time against Maia-3’s 51.9%, and gives the human’s move 25.4% probability on average against 21.3%: a gap of three to four points, against a tie on the full set. A model without a clock input plays exactly the same move distribution at 2 seconds as at 2 minutes; Chessbook-1 knows that people in time trouble play faster and worse, and does the same. Allie 2.0 reads the clock too, and keeps its usual two points over Chessbook-1 here.

It knows how long a human would think

Chessbook-1 does not just pick a move; for every candidate move it also predicts how much of the remaining clock a human would spend on it, as a distribution. The bot samples a think time from that distribution for the move it chose, waits, and then plays. Easy recaptures come instantly. A surprising, hard decision takes a while. Allie 2.0 predicts think time too, one expected time per position rather than per candidate move. This means it may use time unpredictably, like playing a complex tactical move with the speed of an instant recapture, or play a trivial recapture in 20 seconds.

Chessbook-1’s predicted think time matches the humans to within two tenths of a percent of the clock in every rating band (5.4% pooled, humans 5.4%), and it rises with the surprisingness of the move exactly as the humans’ does: about 4% of the clock for an obvious move, 7% for a move the model gave under 1%. Allie 2.0 runs short: 4.9% pooled, 5% to 6% where 2200+ players spend 6% to 7%, and it barely reacts to surprise (5.2% on the moves it gave under 1%, where humans spend 7.3%). In time trouble the same: with under 30 seconds left humans spend 17% of what remains on a move, Chessbook-1 predicts 17%, Allie 14%. Move by move, Chessbook-1 is off by 2.4 points of the clock on average, Allie by 2.6.

It knows who is winning

Chessbook-1 also predicts the result of the game, as win/draw/loss probabilities for the side to move, conditioned on both players’ ratings and the clocks. Maia-3 has a value head too; a rating-blind Stockfish evaluation is the obvious baseline.

The simplest test: in games that ended decisively, how often did the side the model favoured actually win?

Chessbook-1 and Allie 2.0 pick the eventual winner 66–71% of the time, Maia-3 65–69%, and Stockfish’s evaluation 63–66%, in every rating band. The engine is the worst of the four because a +1.5 pawn advantage means something very different for a 900 than for a 2500, and the models know it. Scored properly (the Brier score and log-loss of the win/draw/loss probabilities against the result), Chessbook-1 and Allie are level with each other, Allie a hair ahead in most bands (it reads the whole game), and both are well ahead of Maia-3 everywhere.

This makes for some very cool applications for game review, which I’ve been exploring.

The model and the data: stats for nerds

The data is every rated Lichess game from those twelve months with both players rated 600–3000 and within 400 points of each other, and no bot accounts. The main run used games with a base time of at least two minutes; the fine-tune added bullet and ultrabullet. Positions are sampled so that every rating band contributes equally, the opening is thinned (we do not need a million copies of 1.e4 e5), and the evaluation month is never touched.

Everything here is our own code: the model, the training loop and the serving path are written in Rust on top of the Burn framework, with a forked CUDA backend we spent a chunk of the ten months making fast and correct for multi-GPU training.

Inputs. The board is 64 square tokens. Each square carries piece one-hots, attacker/defender and static-exchange features, mobility, rank and file, castling rights, and decayed from/to marks of the last seven plies per side, plus Maia-style history planes of the previous positions (with 5% dropout so the model also works from a bare position). A global vector carries both ratings as smooth Gaussian bins over 600–2800, the move number, material, and a clock block: 13 log-time bins per clock from 1 s to 4096 s, log base time, log increment, and increment over remaining time. Clocks and history can be missing; a flag says so.

Trunk. 18 pre-norm transformer blocks, 768 wide, 12 heads, SwiGLU MLPs. Two things make it chess-shaped. Every RmsNorm is FiLM-conditioned on the global vector (γ = 1 + Wγ·g, β = Wβ·g), so rating and clock reshape every layer rather than being appended once. And attention uses a Smolgen-style dynamic bias, as in Leela Chess Zero: a small generator compresses the tokens and the globals and emits a 64×64 additive bias per head, so where attention looks depends on the position and on who is playing.

Heads. The policy is a factorised from-square × to-square head (plus promotion piece) over 4,864 moves, through its own transformer block, masked to legal moves and trained with cross-entropy on the human move. A fused auxiliary head predicts the next four plies after the human move (weighted 0.5, 0.25, 0.125, …), which in ablations was worth 0.05 nats at 2200+ for nothing. The value head (W/D/L) and the time head sit on a separate “value block” that reads the trunk detached, so they never push gradients into the policy. The time head is move-aware: it reads the tokens on the move’s from and to squares plus the policy’s surprise for that move and emits the α, β of a Beta distribution over the share of the clock spent. One forward pass returns the policy, the result distribution and a think-time distribution for every candidate move.

Training. Global batch 8,192 (two micro-batches of 512 per GPU on 8×H100), Muon/Aurora for the matrices and AdamW for the rest, a warmup–stable–decay schedule on a 24-hour wall-clock budget (2% warmup, 30% linear decay), EMA weights kept alongside the live ones, 3× oversampling of 2000+ games. ~345M positions. The production run was followed by a 33M-position, 8-hour fine-tune on 8×A100 that trained the value head from ply 1 (the main run ramped it in from ply 10, which made the opening win/draw/loss numbers unreliable) and added bullet and ultrabullet games, which the main run had skipped. Before the production run we did a 14-arm ablation at 512×8 to pick the recipe: history planes, the future-move head, keeping the hand-crafted tactical input features, dropping the engine-supervised heads (they made the model less human on mistakes), batch and learning-rate scaling.

Serving. The bot runs the full model in f16 with CUDA graphs on a single T4 in GKE and answers a prediction in well under a second, then the client waits out the sampled think time before showing the move. A 192×6 “mini” trained on the same architecture provides position embeddings for other Chessbook features.

Benchmark details. Positions are scored with Stockfish 18 MultiPV over every legal move at 8k nodes; a mistake is a move that loses at least 10 points of win probability, a blunder 15, an inaccuracy 5, with evaluations clamped at ±1000 cp for ACPL. Maia-3 is the 79M checkpoint with 7 history positions; Allie 2.0 is the 5.6B MoE run over the full game; Stockfish uses UCI_LimitStrength with UCI_Elo at the mover’s rating, 8 plays per position for a sampled distribution. Noctie, the other commercial human-like bot, has no public weights or API, so it is not in the comparison. The benchmark scripts, the eval sets and the raw per-position outputs for every engine are available on request.

Try it out