Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts

By Justin Grimmer, Brandon M Stewart. 2013

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Missing Data Imputation for Supervised Learning (2016, background): “de the estimation of heterogeneous treatment eﬀects (Imai and Strauss, 2011; Green and Kern, 2012; Imai et al., 2013; Grimmer et al., 2014), text analysis (Quinn et al., 2010; Hopkins and King, 2010; Grimmer and Stewart, 2013; Lauderdale and Clark, 2014; Wilkerson et al., 2015), and record linkage (”
Computer-Assisted Text Analysis for Comparative Politics (2015, methodology): “Unlike the issues involved in multilingual text processing, these methods have been well developed elsewhere (e.g., Grimmer and Stewart 2013).”
Discovery of Treatments from Text Corpora (2016, methodology): “We selected ten treatments using a combination of guidance from the sIBP, assessment using CE scores, and our own qualitative assessment of the models (Grimmer and Stewart, 2013).”
Directions in abusive language training data, a systematic review: Garbage in, garbage out (2020, background): “Traditional qualitative methods cannot scale to handle the hundreds of millions of posts which appear on each major social media platform every day, and introduce various inconsistencies and biases [8].”
Election Campaigning on Social Media: Politicians, Audiences, and the Mediation of Political Communication on Facebook and Twitter (2018, methodology): “Recent advances in quantitative text analysis (Grimmer & Stewart, 2013) enable us to perform text analysis at a larger scale.”
How to make causal inferences using texts (2018, unknown): “While validation approaches may vary—this necessarily involves reading documents ( 15 , 22 , 29 )—it is helpful to fix a set of human-assigned labels, example documents, and automated keyword labels in advance to avoid subtle influence from the test set.”
Framing Climate Change: Economics, Ideology, and Uncertainty in American News Media Content From 1988 to 2014 (2019, background): “These techniques are increasingly used by scholars in the social sciences to study news content and political discourse more broadly (Young and Soroka, 2012; Grimmer and Stewart, 2013; Lacy et al., 2015).”
Third wave positive psychology: broadening towards complexity (2020, background): “Still, such data bring numerous challenges, and while computational social science can complement other approaches, they do not replace them (Grimmer & Stewart, 2013; Kern, Williams et al., 2018; Iliev et al., 2014).”
Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology (2018, methodology): “Compared with other topic-clustering methods, a further advantage of LDA topic modeling is its mixed membership approach (Grimmer & Stewart, 2013, p. 18), i.e., one document can contain several topics, which is a useful assumption.”
Quantitative analysis of large amounts of journalistic texts using topic modelling (2016, methodology): “For a general overview of the available methods and pitfalls for topic models, see Grimmer and Stewart (2013).”
Twitter as Data (2018, background): “Computationally analyzing text is a growing field in the social sciences (Grimmer and Stewart, 2013; Lucas et al., 2015), and Twitter provides scholars with large corpora.”
Measuring Polarization with Text Analysis: Evidence from the UK House of Commons, 1811–2015 (2019, methodology): “Unsupervised methods come with significant post hoc validation costs, as the researcher “must combineexperimental, substantive, and statistical evidence todemonstrate that themeasures are as conceptually valid as measures from an equivalent supervised model” (Grimmer and Stewart 2013, 5).”
Understanding the Political Ideology of Legislators from Social Media Images (2019, background): “In more recent work, however, there are been greater interest in developing more general methods for measuring political ideology (Grimmer and Stewart, 2013, Iyyer et al., 2014, Sim et al., 2013, Slapin and Proksch, 2008).”
Topic Modeling in Management Research: Rendering New Theory from Textual Data (2019, background): “In the current emergent phase of its employment, scholars are still learning the best ways to reveal constructs and develop theory (Evans & Aceves, 2016; Grimmer & Stewart, 2013)—which implies a need for deeper insights into how topic modeling can inform new theories.”
Simultaneously Discovering and Quantifying Risk Types from Textual Risk Disclosures (2014, methodology): “They are valuable since they could identify organizations of texts that are theoretically useful but perhaps understudied or previously unknown (Grimmer and Stewart 2013).”