Expert Political Judgment : How Good Is It ? How Can We Know ?

By Scott Armstrong. 2012

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Social Sciences & Humanities

Citations in the corpus

When Will AI Exceed Human Performance? Evidence from AI Experts (2017, background): “Why think AI experts have any ability to foresee AI progress? In the domain of political science, a long-term study found that experts were worse than crude statistical extrapolations at predicting political outcomes [14].”
Simple Rules for Complex Decisions (2017, background): “A large body of work shows that intuitive judgments are generally inferior to those based on statistical models [4, 5, 19, 20, 33].”
Forecasting in social settings: The state of the art (2020, background): “Judgmental predictions have been evaluated by Tetlock (2006) who has compared the forecasts of experts in different macroeconomic fields to forecasts made by well-informed laity or those based on simple extrapolation from current trends.”
Elo Ratings and the Sports Model: A Neglected Topic in Applied Probability? (2017, background): “Note it is rootmean-square, and so is directly interpretable as typical additive error in estimating a probability.”
Integrating explanation and prediction in computational social science (2021, background): “Activities in this quadrant encompass time series modelling 43 , prediction contests 44 , and much of supervised machine learning 10 , ranging from simple linear regression to complex artificial neural networks.”
Simple rules to guide expert classifications (2020, unknown): “E-mail: scgoel@stanford.edu Dawes, 1979; Dawes et al., 1989; Camerer and Johnson, 1997; Tetlock, 2005; Kleinberg et al., 2015, 2017).”
Wisdom or Madness? Comparing Crowds with Expert Evaluation in Funding the Arts (2016, unknown): “For example, studies on forecasting have supported the idea that experts are no more accurate than informed amateurs (Tetlock, 2005), while examination of Wikipedia has shown that it compares favorably on many dimensions to expert-created content (Clauson, Polen, Boulos, & Dzenowagis, 2008; Giles,…”
The Behavioral Revolution and International Relations (2017, unknown): “Tetlock 2005; Tetlock and Gardner 2015.”
Obliquity (2010, unknown): “The political scientist, Philip Tetlock (2005), studied the political predictions of experts over two decades.”
To judge a book by its weight you need to know its content: Knowledge moderates the use of embodied cues (2012, result): “In contrast, research into confirmatory hypothesis testing has shown that experts are as likely to search for supporting evidence as novices (Tesser & Leone, 1977; Tetlock, 2005).”
Effects of Corporate Social Responsibility and Irresponsibility Policies (2013, unknown): “That conclusion was reinforced by a study of more than 82,000 judgmental forecasts made over 20 years by 284 experts in politics and economics (Tetlock, 2005).”
When overconfidence is revealed to others: Testing the status-enhancement theory of overconfidence (2013, background): “This could provide one explanation for why experts update their judgments less often and more slowly than one might expect (Tetlock, 1998; Tetlock, 2005).”
The Cognitive Revolution and the Political Psychology of Elite Decision Making (2013, background): “70 Tetlock 2005.”
Exploring the filter bubble: the effect of using recommender systems on content diversity (2014, background): “Tetlock [16], a political scientist, ran a study in which he asked diﬀerent people with various background for opinions on political and economic issues.”
What Motivates Effort?: Evidence and Expert Forecasts (2018, unknown): “2There is a larger literature on forecasting about topics other than research results, e.g., the Good Judgment Project on national security (Tetlock, 2010 and Tetlock and Gardner, 2015).”
Compromising Talent: Issues in Identifying and Selecting Talent in Sport (2018, background): “Importantly, research suggests ascribing rigidly to one theory or way of thinking is associated with decreased accuracy in predictions (Tetlock, 2005).”