The rating of chessplayers, past and present

By A. Elo. 1978

Subfields: Statistics and Probability, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Game & Decision Theory

Citations in the corpus

Deep Reinforcement Learning from Human Preferences (2017, background): “It can be understood as equating rewards with a preference ranking scale analogous to the famous Elo ranking system developed for chess (Elo, 1978).”
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models (2020, methodology): “To evaluate the modellevel quality of NLG models based on pairwise comparison in sample-level, we adopt the skill rating system similar to ELO (Elo 1978) and Trueskill (Herbrich et al. 2007), which is a method for assigning a numerical skill to players in a player-vs-player game, given a win-loss…”
Reward learning from human preferences and demonstrations in Atari (2018, background): “It can be interpreted as equating rewards with a preference ranking scale analogous to the Elo ranking system developed for chess (Elo, 1978).”
Improving alignment of dialogue agents via targeted human judgements (2022, background): “Response preference data (section 2.3) allows us to train a Preference RM that for each response predicts an Elo preference score such that the softmax over the scores predicts the preference probability, following (Elo, 1978; Menick et al., 2022; Stiennon et al., 2020).”
QLoRA: Efficient Finetuning of Quantized LLMs (2023, methodology): “We randomly sample from the set of labeled comparisons to compute Elo [16, 17].”
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators (2024, methodology): “At the end, the head-to-head comparisons are converted to an Elo rating Elo & Sloan (1978) which serves as the model score.”
Emergent Tool Use From Multi-Agent Autocurricula (2019, background): “Metrics like ELO [63] or Trueskill [64] can more reliably measure whether performance is improving relative to previous policy versions or other policies in a population; however, these metrics still do not give insight into whether improved performance stems from new adaptations or improving previo”
Human-level performance in 3D multiplayer games with population-based reinforcement learning (2018, methodology): “We estimate skill using the Elo rating system (15)”
A survey and critique of multiagent deep reinforcement learning (2018, methodology): “The Elo rating system [90] was originally devised to rate chess player skills,13 TrueSkill [138] extended Elo by tracking uncertainty in skill rating, supporting draws, and matches beyond 1 vs 1; α−Rank is a more recent alternative to ELO [243].”
Emergent Coordination Through Competition (2019, methodology): “B.1 FITNESS We use Elo rating (Elo (1978)), introduced to evaluate the strength of human chess players, to measure an agent’s performance within the population of learning agents and determine eligibility for evolution.”
Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Games (2021, methodology): “It can be found θ (cid:63) ← Train π t − i against mixture of j σ π with n gradient steps 8: Compute missing entries in the evaluation matrix A P i × P − i 10: end for 11: Output: Nash value on A P i × P − i that models generated by PSRO w. BD&RD outperform other methods and reaches an Elo score of ”
Skill Rating for Generative Models (2018, methodology): “In games such as chess or tennis, skill rating systems such as Elo [Elo, 1978] or Glicko2 [Glickman, 2013] evaluate players by observing a record of wins and losses of multiple players and inferring the value of a latent, unobserved skill variable for each player that explains the records of wins…”
Deliberate practice: Is that all it takes to become an expert? (2014, unknown): “001 working memory capacity—the ability to maintain information in a highly active state (Engle, 2002). Meinz and Hambrick (2010) found that although deliberate practice accounted for nearly half (45.1%) of the variance in pianists' performance on a sight-reading task, working memory capacity accoun”
Eliminating Supportive Crowds Reduces Referee Bias (2021, background): “Garicano et al. (2005) and Sutter and Kocher (2004) also identified a systematic bias among referees in the top levels of Spanish and German football, respectively.”
Adversarial Diffusion Distillation (2023, methodology): “To calculate rankings when comparing more than two models based on 1v1 comparisons we use ELO Scores (higher-is-better) [10] which were originally proposed as a scoring method for chess players but have more recently also been applied to compare instruction-tuned generative LLMs [1, 2].”
The role of deliberate practice in chess expertise (2005, methodology): “…elite players in the sample had international ratings so for those providing national federation ratings, we converted these ratings to Elo ratings (Elo, 1986) using native rating levels for German (first converting any older INGO ratings that predated the modern rating system using suggested…”
DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess (2016, unknown): “The results also show that DeepChess is over 60 Elo [7] stronger than Crafty , a program which has won two WCCCs and has been manually tuned for thirty years.”
From Motor Control to Team Play in Simulated Humanoid Football (2021, methodology): “Performance of each agent in the population in the full game of football was measured by playing 64 matches against the evaluation agents, at regular intervals during training, and computing Elo scores (69).”
COMPUTING “ELO RATINGS” OF MOVE PATTERNS IN THE GAME OF GO (2007, methodology): “The BradleyTerry model is a pairwise comparison model based on the logit link function, and is the theoretical basis of the Elo rating system (Elo, 1978).”
EDGE: Editable Dance Generation From Music (2022, methodology): “In addition to computing the raw win rate of our method when evaluated directly against competing methods shown in the table, we also computed each method’s Elo rating, which, unlike flat win rate, is able to simultaneously capture the relative quality of more than 2 models [9].”