Large Language Monkeys: Scaling Inference Compute with Repeated Sampling

By Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher R'e, Azalia Mirhoseini. arXiv.org, 2024

866 citations recorded by the source corpus.

Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit models to making only one attempt at a problem. Here, we explore inference compute as another axis for scaling, using the simple technique of repeatedly sampling candidate solutions from a model. Across multiple tasks and models, we observe that coverage -- the fraction of problems that are solved by any generated sample -- scales with the number of samples over four orders of magnitude. Interestingly, the relationship between coverage and the number of samples is often log-linear and can be modelled with an exponentiated power law, suggesting the existence of inference-time scaling laws. In domains like coding and formal proofs, where answers can be automatically verified, these increases in coverage directly translate into improved performance. When we apply repeated sampling to SWE-bench Lite, the fraction of issues solved with DeepSeek-Coder-V2-Instruct increases from 15.9% with one sample to 56% with 250 samples, outperforming the single-sample state-of-the-art of 43%. In domains without automatic verifiers, we find that common methods for picking from a sample collection (majority voting and reward models) plateau beyond several hundred samples and fail to fully scale with the sample budget.

View Paper: https://www.semanticscholar.org/paper/f66b6049946a10080d1f2a7ee6a40e5cca3ee6a0

Citations in the corpus

• Generative Verifiers: Reward Modeling as Next-Token Prediction (2024, unknown): “…discriminative RMs miss out on the inherent strengths of generative LLMs, such as unified instruction tuning (Chung et al., 2022), chain-of-thought (CoT) reasoning (Wei et al., 2022), and utilizing additional inference-time computation for better performance (Wang et al., 2022; Brown et al., 2024).”
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (2025, unknown): “Following (Brown et al., 2024), we manually inspect all CoTs that led to correct answers on the most challenging solvable problems in the GSM8k dataset—those with an average accuracy below 5% but above 0%.”
• LIMO: Less is More for Reasoning (2025, unknown): “This can be achieved by augmenting LLMs with methods such as parallel sampling (Brown et al., 2024; Wang et al., 2022; Li et al., 2022) or symbolic tree search (Hao et al., 2023; Chen et al., 2024; Yao et al., 2023) to enhance reasoning ability.”
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking (2025, unknown): “Although rejection sampling methods [Yuan et al., 2023, Brown et al., 2024] can improve data quality, they do not guarantee correct intermediate steps.”
• s1: Simple test-time scaling (2025, unknown): “To enhance models’ performance on reasoning-related tasks, researchers have pursued several strategies: Prior works have explored continuing training language models on specialized corpora related to mathematics and science (Azerbayev et al., 2023; Yang et al., 2024), sometimes even synthetically…”
• Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs (2025, unknown): “These approaches range from simple parallel sampling (Li et al., 2024; Brown et al., 2024) to more sophisticated methods using verifiers or process reward models (PRMs) (Lightman et al., 2023; Yao et al., 2023).”