Scaling Scaling Laws with Board Games
By Andrew Jones. 2021
64 citations recorded by the source corpus.
The largest experiments in machine learning now require resources far beyond the budget of all but a few institutions. Fortunately, it has recently been shown that the results of these huge experiments can often be extrapolated from the results of a sequence of far smaller, cheaper experiments. In this work, we show that not only can the extrapolation be done based on the size of the model, but on the size of the problem as well. By conducting a sequence of experiments using AlphaZero and Hex, we show that the performance achievable with a fixed amount of compute degrades predictably as the game gets larger and harder. Along with our main result, we further show that the test-time and train-time compute available to an agent can be traded off while maintaining performance.
Subfields: Artificial Intelligence, Clinical Psychology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Clinical Psychology
Citations in the corpus
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (2024, unknown): “Jones [33] finds scaling laws in the context of the board game Hex, observing that performance scales predictably with model size and the difficulty of the problem.”
- Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs (2025, unknown): “This suggests that training helps the model learn to use limited token budgets more efficiently — more train-time interactions lead to more efficient test-time performance (Jones, 2021).”
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (2024, methodology): “The tradeoff between train-time and test-time compute using Monte-Carlo tree search applied to the board game Hex was previously studied by Jones [16].”