Teaching a neural network to judge Azul
1.5 million self-play positions, a network trained from scratch in Go and again in PyTorch on a GPU, and the one feature that turned a tie into a win.
The classic engine judges positions with fifteen terms a human chose and self-play tuned. The machine-learning question: can a model learn a better judgement from the engine's own games, with no Azul knowledge written in?
The data pipeline
A new command, azul gendata, plays the engine against itself on every CPU core. 30,000 games gave 1.55 million positions in seven minutes. Each position becomes 170 small integers seen from the side to move: walls, pattern lines, floors, scores and the tiles left on the table. Each carries three labels: the search's value, the final score margin and who won. Five percent of moves are random, so games do not all follow the same habits.
The model, twice
The model is a small multilayer perceptron: 170 inputs, two hidden ReLU layers, one output predicting the final margin in points. I trained it two ways on the same data, split and target:
- From scratch in Go, with no libraries: forward pass, backpropagation, Adam and data-parallel minibatches over CPU cores, about 300 lines. A gradient-check test compares backpropagation against numerical derivatives.
- In PyTorch, on an RTX 5070 with CUDA, as a benchmark and a cross-check.
| Go, CPU | PyTorch, GPU | |
|---|---|---|
| Training, 64 × 16 network | 19 s | 24 s |
| Training, 128 × 32 network | 42 s | 22 s |
| Validation error (points²) | 153.0 | 153.2 |
The two agree to the decimal, which also proves the hand-written training. For a network this small the GPU does not help: each step is too little work to outweigh moving the batch.
What the model sees decides everything
The first network, given only the raw board, tied the classic engine: −7 ± 34 Elo. Then I added one group of features: the board as it would look if the round were scored now, which the classic evaluation also uses. Same network, same data size: +141 ± 13 Elo at equal positions searched. Feature engineering beat model size.
The training target mattered too. Learning only the search's opinion, or only who won, both did worse than half and half (−127 and −84 Elo against −7 on the raw board).
Strength at equal time
A network judges better but costs more per position. At 40 ms per move the 64 × 16 network beat the classic engine by +64 ± 13 Elo over 3,000 games, and the bigger 128 × 32 network, though it knew more, was too slow to win at all. Making the network fast became the next problem; see the next post.
All numbers are reproducible with the commands in the technical write-up.