A Method of Automated Nonparametric Content Analysis for Social Science

By Daniel Hopkins, Gary King. 2010

820 citations recorded by the source corpus.

The increasing availability of digitized text presents enormous opportunities for social scientists. Yet hand coding many blogs, speeches, government records, newspapers, or other sources of unstructured text is infeasible. Although computer scientists have methods for automated content analysis, most are optimized to classify individual documents, whereas social scientists instead want generalizations about the population of documents, such as the proportion in a given category. Unfortunately, even a method with a high percent of individual documents correctly classified can be hugely biased when estimating category proportions. By directly optimizing for this social science goal, we develop a method that gives approximately unbiased estimates of category proportions even when the optimal classifier performs poorly. We illustrate with diverse data sets, including the daily expressed opinions of thousands of people about the U.S. presidency. We also make available software that implements our methods and large corpora of text for further analysis.

Subfields: Artificial Intelligence, General Social Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

Citations in the corpus

Missing Data Imputation for Supervised Learning (2016, background): “upervised learning include the estimation of heterogeneous treatment eﬀects (Imai and Strauss, 2011; Green and Kern, 2012; Imai et al., 2013; Grimmer et al., 2014), text analysis (Quinn et al., 2010; Hopkins and King, 2010; Grimmer and Stewart, 2013; Lauderdale and Clark, 2014; Wilkerson et al., 201”
Computer‐Assisted Keyword and Document Set Discovery from Unstructured Text (2017, background): “See for example Eshbaugh-Soha (2010), Gentzkow and Shapiro (2010), Hopkins and King (2010), Ho and Quinn (2008), King, Pan and Roberts (2013), Puglisi and Snyder (2011).”
How to make causal inferences using texts (2018, neutral): “hat determines how coders should put documents into the categories of interest. More recently, a wide array of machine learning methods are used to discover gfrom the data (Blei, Ng and Jordan, 2003; Hopkins and King, 2010). These newly discovered categories can help shape research questions, identi”
Every tweet counts? How sentiment analysis of social media can improve our knowledge of citizens’ political preferences with an application to Italy and France (2013, methodology): “Accordingly, we will employ the method recently proposed by Hopkins and King (2010).”
Structural Topic Models for Open‐Ended Survey Responses (2014, background): “Second, it allows analysts to do this while studying how the prevalence and content of topics change with information that is particular to each respondent (e.g., whether the respondent received the treatment or background demographic data).”
A Meta-Analysis of State-of-the-Art Electoral Prediction From Twitter Data (2012, methodology): “Jungherr et al. (2011) later criticized some of the decisions taken by Tumasjan et al.”
Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology (2018, background): ““ Preprocessing text strips out information, in addition to reducing complexity, but experience in this literature is that the trade-off is well worth it ” (Hopkins & King, 2010, p. 223).”
Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It (2017, background): “In the supervised case, which is relatively rare in political science applications (though see e.g. Laver, Benoit and Garry, 2003; Hopkins and King, 2010; Diermeier et al., 2011; D’Orazio et al., 2014), researchers have a set of hand-labeled training documents and they wish to learn the…”
How Censorship in China Allows Government Criticism but Silences Collective Expression (2013, methodology): “We thus adapt to the Chinese language the methodology introduced in the English language by Hopkins and King (2010).”