TrueSkillTM: A Bayesian Skill Rating System

By Ralf Herbrich, Tom Minka, Thore Graepel. 2006

710 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Findings of the 2014 Workshop on Statistical Machine Translation (2014, background): “TrueSkill is an adaptive, online system that employs a similar model of relative ability Herbrich et al. (2006).”
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models (2020, methodology): “To evaluate the modellevel quality of NLG models based on pairwise comparison in sample-level, we adopt the skill rating system similar to ELO (Elo 1978) and Trueskill (Herbrich et al. 2007), which is a method for assigning a numerical skill to players in a player-vs-player game, given a win-loss re”
Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge (2019, methodology): “Introducing the data-efficient TrueSkill algorithm (Herbrich, Minka, and Graepel 2006; Sakaguchi, Post, and Van Durme 2014) to NLG.”
TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations (2021, neutral): “To further evaluate the performance of different algorithms, we rank all the algorithms with the TrueSkill rating system (Herbrich, Minka, and Graepel, 2006), and the results are shown in the last row of Table 1 and Figure 3(b).”
Emergent Tool Use From Multi-Agent Autocurricula (2019, background): “[64] Ralf Herbrich, Tom Minka, and Thore Graepel.”
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments (2017, methodology): “To do so, we employ TrueSkill [21]—a metric similar to the ELO rating, but more popular in 1-vs-1 competitive video-games.”
Findings of the 2017 Conference on Machine Translation (WMT17) (2017, methodology): “From these rankings, we produced pairwise translation comparisons, and applied the TrueSkill algorithm (Herbrich et al., 2007; Sakaguchi et al., 2014) to produce system rankings.”
RankME: Reliable Human Ratings for Natural Language Generation (2018, methodology): “Finally, our results suggest that TrueSkill is a cost-effective alternative for producing overall relative rankings of multiple systems.”
Findings of the E2E NLG Challenge (2018, methodology): “The final evaluation results were produced using the TrueSkill algorithm (Herbrich et al., 2006; Sakaguchi et al., 2014), with partial ordering into significance clusters computed using bootstrap resampling (Bojar et al.”
A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agents (2019, methodology): “, 2016), we use Trueskill (Herbrich et al., 2007) to cluster systems into equivalently rated groups according to their partial relative”
Efficient Online Scalar Annotation with Bounded Support (2018, methodology): “Further details are provided by Herbrich et al. (2006).8 Another important property of TrueSkill is “match quality (chance to draw)”.”
Crowdsourced Data Management: A Survey (2016, neutral): “TrueSkill [53].”
Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine (2010, methodology): “We will not provide a detailed derivation of the inference equations because adPredictor can be seen as a special case of the TrueSkill™ rating algorithm for games ( Herbrich, Minka, & Graepel, 2007 ).,This appears to be a rather general phenomenon, and a similar phenomenon has been described in ( H”
Multimodal Language Analysis with Recurrent Multistage Fusion (2018, methodology): “These use Bayesian ranking algorithms (Herbrich et al., 2007) to model both person-independent and person-dependent features (Liang et al.”
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale (2019, neutral): “8 NBA players [18], exploratory analysis of 1.7×106 taxi trajectories [36], and probabilistic modeling for processing hundreds-of-thousands of Xbox live games per day to rank and match players fairly [33, 55]. In all these large-scale programs, despite the number of observations being large, model s”