The Effect of Sampling Temperature on Problem Solving in Large Language Models

By Matthew Renze, Erhan Guven. Conference on Empirical Methods in Natural Language Processing, 2024

328 citations recorded by the source corpus.

In this research study, we empirically investigate the effect of sampling temperature on the performance of Large Language Models (LLMs) on various problem-solving tasks. We created a multiple-choice question-and-answer (MCQA) exam by randomly sampling problems from standard LLM benchmarks. Then, we used nine popular LLMs with five prompt-engineering techniques to solve the MCQA problems while increasing the sampling temperature from 0.0 to 1.6. Despite anecdotal reports to the contrary, our empirical results indicate that changes in temperature from 0.0 to 1.0 do not have a statistically significant impact on LLM performance for problem-solving tasks. In addition, these results appear to generalize across LLMs, prompt-engineering techniques, and problem domains. All code, data, and supplemental materials are available on GitHub at: https://github.com/matthewrenze/jhu-llm-temperature

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/c9b6d0db6c2bebe0bcf593d95bea0e62b2443ef3

Citations in the corpus

• ReservoirChat: Interactive Documentation Enhanced with LLM and Knowledge Graph for ReservoirPy (2025, unknown): “For every model a temperature of 0.1 was used to minimize stochastic effects, as preliminary tests showed it was a good compromise and close to a temperature of 0 2 which is often preferred [27].”