Speed is strength: an NNUE-style evaluator in Go
Updating only what a move changes, in exact integer arithmetic, made the network three times faster and worth 50 Elo more, with the same weights.
After training, the network judged positions better than the hand-written evaluation but took 2,161 ns per searched position against 309. In a game engine that matters: with the same thinking time, a slower judge sees fewer positions. Chess engines solved this with NNUE, an efficiently updatable neural network. This post applies the same idea in plain Go.
Measure first
A benchmark that runs the real search, not one evaluation in a loop, showed where the time went: 83% in the network's forward pass, 15% turning the position into features. The search evaluates positions that differ by a move or two, sibling after sibling, yet every evaluation recomputed the first layer from scratch.
Three changes
- Incremental first layer. The evaluator keeps the first layer's sums from its last call and applies only the rows of inputs that changed, usually a handful out of 170. If more than 24 changed, a fresh pass is cheaper and it does that instead.
- Exact integers. Floating-point sums drift after thousands of updates. The first layer runs in fixed-point integers, with the input scaling folded into the weights, so an update is exact. A test walks thousands of positions and checks the incremental result equals a fresh one every time, and stays within 0.02 points of the float network.
- Sparse second layer. After ReLU about half the first-layer units are zero, so the second layer reads only the active ones, with its weights transposed to make that a straight run through memory.
Results
| ns per position | vs classic, 40 ms per move | |
|---|---|---|
| Classic evaluation | 309 | |
| Network, first version | 2,161 | +64 ± 13 Elo |
| Network, incremental | 704 | +112 ± 13 Elo |
The same weights, 3.1 times faster, worth about 50 Elo: the network now takes 66% of the points against the hand-tuned engine over 3,000 games. The speed-up also rescued the bigger network, from +6 to +135 at equal time. Head to head it ties the small one (−10 ± 12 over 3,000 games), so the board keeps the lighter 64 × 16 network, which also runs faster in the browser.
Next: the network plays itself, a new network learns from those games, and each generation has to beat its parent. That loop is running now on a remote server.