Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

By Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar. arXiv.org, 2024

1,973 citations recorded by the source corpus.

Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this paper, we study the scaling of inference-time computation in LLMs, with a focus on answering the question: if an LLM is allowed to use a fixed but non-trivial amount of inference-time compute, how much can it improve its performance on a challenging prompt? Answering this question has implications not only on the achievable performance of LLMs, but also on the future of LLM pretraining and how one should tradeoff inference-time and pre-training compute. Despite its importance, little research attempted to understand the scaling behaviors of various test-time inference methods. Moreover, current work largely provides negative results for a number of these strategies. In this work, we analyze two primary mechanisms to scale test-time computation: (1) searching against dense, process-based verifier reward models; and (2) updating the model's distribution over a response adaptively, given the prompt at test time. We find that in both cases, the effectiveness of different approaches to scaling test-time compute critically varies depending on the difficulty of the prompt. This observation motivates applying a"compute-optimal"scaling strategy, which acts to most effectively allocate test-time compute adaptively per prompt. Using this compute-optimal strategy, we can improve the efficiency of test-time compute scaling by more than 4x compared to a best-of-N baseline. Additionally, in a FLOPs-matched evaluation, we find that on problems where a smaller base model attains somewhat non-trivial success rates, test-time compute can be used to outperform a 14x larger model.

Subfields: Aerospace Engineering, Electrical and Electronic Engineering, Nuclear and High Energy Physics

View Paper: https://www.semanticscholar.org/paper/8292083dd8f6ae898ea0ee54a6b97997d1a51c9d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Electrical and Electronic Engineering

In Nuclear and High Energy Physics

Citations in the corpus

LlaVA-CoT: Let Vision Language Models Reason Step-By-Step (2024, positive): “Our LLaVA-CoT facilitates a progressive, step-by-step reasoning process that enhances the reasoning capabilities of Vision-Language Models (VLMs) and allows for effective inference time scaling [50].”
LIMO: Less is More for Reasoning (2025, neutral): “Instead of focusing on scaling model parameters and training data (Kaplan et al., 2020), recent work has shifted to exploring test-time scaling (OpenAI, 2024; Snell et al., 2024), i.e., increasing the number of tokens to improve performance.”
Training Large Language Models to Reason in a Continuous Latent Space (2024, positive): “One promising direction is pretraining LLMs with continuous thoughts, which may enable models to generalize more effectively across a wider range of reasoning scenarios.”
s1: Simple test-time scaling (2025, neutral): “We classify test-time scaling methods into 1) Sequential , where later computations depend on earlier ones (e.g., a long reasoning trace), and 2) Parallel , where computations run independently (e.g., majority voting) (Snell et al., 2024; Brown et al., 2024).”
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models (2025, positive): “Test-Time Scaling has been shown to be an effective method to enhance the reasoning abilities of LLMs and MLLMs [108, 94, 87, 70, 120, 36, 152, 125].”
InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025, positive): “Test-time scaling (TTS) has been empirically demonstrated as an effective approach to enhance the reasoning capabilities of LLMs and MLLMs, particularly for complex tasks that require multi-step inference [65, 82, 113, 143, 179].”
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training (2025, positive): “Subsequent work (Zelikman et al., 2022; Feng et al., 2023; Tian et al., 2024; Chen et al., 2024a; Snell et al., 2024) has demonstrated that fine-tuning verifiers during inference improves model accuracy, effectively utilizing test-time computation.”
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, neutral): “During training, they adopt LLM-as-a-judge to score the synthetic responses and set a threshold to collect high-quality query-response pairs for the subsequent training iteration.”
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, positive): “This approach consistently improves performance on challenging math and code problems that require reasoning by increasing the computational resources allocated during inference [5,77].”
rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking (2025, neutral): “Scaling Test-time Compute has introduced new scaling laws, allowing LLMs to improve performance across by generating multiple samples and using reward models for best-solution selection [Snell et al., 2024, Wu et al., 2024, Brown et al., 2024].”
ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (2025, positive): “Building upon this foundation, recent research has shifted focus from train-time scaling to test-time scaling [17], where additional computational resources are allocated during inference to enable the generation of intermediate reasoning steps.”
Kimi k1.5: Scaling Reinforcement Learning with LLMs (2025, positive): “To further enhance the model’s reasoning capabilities, planning algorithms are employed to explore various thought processes, generating improved CoT at inference time (Yao et al. 2024; Y. Wu et al. 2024; Snell et al. 2024).”
L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning (2025, neutral): “Formally, given an input prompt x and a target length n gold , the model is expected to generate a response y whose length n y minimizes the absolute difference | n gold − n y | while simultaneously producing the correct answer.”
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (2023, neutral): “Next, in line 4, we determine whether the majority of the responses jailbreak the LLM. Finally, in line 5, we select a response uniformly at random that is consistent with the majority vote found in the previous line, and return that response.”
Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs (2025, positive): “Snell et al. (2024) systematically explore this space by developing various methods to search through reasoning trajectories.”
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024, positive): “…[220, 232, 5, 172, 157, 210], scaling vision encoders [252, 66, 36, 293, 185] and language models [231, 235, 64, 19, 229, 221, 62], incorporating more diverse and high-quality datasets [124, 234, 25, 155], and refining the test-time scaling process [215, 249, 230] to boost performance.”
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (2025, positive): “In conclusion, while PRM demonstrates a good ability to rerank the top-N responses generated by the model or assist in guided search (Snell et al., 2024), its advantages are limited compared to the additional computational overhead it introduces during large-scale reinforcement learning process in…”
Dream 7B: Diffusion Large Language Models (2025, neutral): “This timestep-based approach introduces an additional dimension for inference-time scaling (Snell et al., 2024; Muennighoff et al., 2025; Geiping et al., 2025), which works alongside existing techniques such as chain-of-thought reasoning used in large language models like OpenAI o1 (Jaech et al.,…”
Training Language Models to Self-Correct via Reinforcement Learning (2024, neutral): “Example:even_odd_count( − 12) ==> (1, 1) even_odd_count(123) ==> (1, 2) """ even_count = 0 odd_count = 0 for digit in str(abs(num)): int_digit = int(digit) if int_digit % 2 == 0: even_count += 1 else: odd_count += 1 return even_count, odd_count”
ReasonIR: Training Retrievers for Reasoning Tasks (2025, unknown): “At the same time, challenging benchmarks for reasoning-intensive retrieval have emerged (Xiao et al., 2024a; Wang et al., 2024; Su et al., 2024; Enevoldsen et al., 2025), yet high-performing models remain scarce.”