Quantitative Analysis of Culture Using Millions of Digitized Books
By Erez Lieberman Aiden, Jean-Baptiste Michel. 2010
2,863 citations recorded by the source corpus.
Linguistic and cultural changes are revealed through the analyses of words appearing in books. We constructed a corpus of digitized texts containing about 4% of all books ever printed. Analysis of this corpus enables us to investigate cultural trends quantitatively. We survey the vast terrain of ‘culturomics,’ focusing on linguistic and cultural phenomena that were reflected in the English language between 1800 and 2000. We show how this approach can provide insights about fields as diverse as lexicography, the evolution of grammar, collective memory, the adoption of technology, the pursuit of fame, censorship, and historical epidemiology. Culturomics extends the boundaries of rigorous quantitative inquiry to a wide array of new phenomena spanning the social sciences and the humanities.
Subfields: Artificial Intelligence, Cultural Studies, Language and Linguistics, Computational Linguistics & Semantics, Social Sciences & Humanities
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yoshua Bengio
- Percy Liang
- Jianfeng Gao
- Noah A. Smith
- Yejin Choi
- Ruslan Salakhutdinov
- Luke S. Zettlemoyer
In Cultural Studies
- Stephan Mandt
- Robert Bamler
- Bryan Perozzi
- Rami Al-Rfou
- Vivek Kulkarni
- Steven S. Skiena
- Josh Tenenbaum
- Charles Kemp
In Language and Linguistics
- Kie Zuraw
- Bruce Hayes
- Charles Kemp
- N. Schmitt
- Stephan Mandt
- Ali Farhadi
- Santosh Kumar Divvala
- Marc Brysbaert
In Computational Linguistics & Semantics
- Stephan Mandt
- Ali Farhadi
- Santosh Kumar Divvala
- Jacob Eisenstein
- Xiaochuang Han
- Robert Bamler
- Yang Xu
- Rami Al-Rfou
In Social Sciences & Humanities
- Stephan Mandt
- Ali Farhadi
- Santosh Kumar Divvala
- Jacob Eisenstein
- Xiaochuang Han
- Robert Bamler
- Yang Xu
- Yi Xu
Citations in the corpus
- How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007) (2015, background): “Cobb (2007), in a study using corpus coverage to calculate learning opportunities, showed that learners’ typically small vocabulary sizes of 2,500–3,000 1 Michel et al. (2011) found 1,022,000 individual word forms in a computer analysis of a corpus of 5,195,769 digitized books.”
- Variational Beam Search for Novelty Detection (2021, methodology): “Our first dataset is the Google Books corpus (Michel et al., 2011) consisting of n -grams, which is sufficient for learning word embeddings.”
- Reading Into the Future: Competence for the 21st Century (2012, background): “With regard to printed material, for example, it was estimated that nearly 125 million books were in print by the beginning of 2011, with 15 million available digitally (Michel et al., 2011).”
- News events prediction using Markov logic networks (2018, background): “[23] developed ‘culturomics’, an approach to studying how culture develops in societies, by applying text mining techniques to millions of digitalised books.”
- Intersecting constraint families: An argument for harmonic grammar (2017, methodology): “We used the Google n-grams corpus for French (Michel et al. 2011, Lin et al. 2012), which includes only sequences that occur at least forty times in the Google Books corpus; we limited results to publications since 1900.”
- Long-Term Trends in the Public Perception of Artificial Intelligence (2016, methodology): “For example, by analyzing 200 years of data from Google Books, it is possible to quantify the adoption of new technologies or changes in psychological attitudes through linguistic patterns (Michel et al. 2011; Greenfield 2013).”
- Syntactic Annotations for the Google Books NGram Corpus (2012, methodology): “The sheer quantity of and broad historical scope of the data has enabled a wide range of analyses (Michel et al., 2011; Ravallion, 2011).”
- Harm inflation: Making sense of concept creep (2020, methodology): “This analysis was supported by quantitative analyses of changes in the relative frequency of related words in the Google Books corpus, a demonstration of “culturomics” (Michel et al., 2011).”
- VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases (2015, methodology): “Our relation phrases were gathered using the Google Books Ngram (English 2012) corpus [26].”
- Towards a sociology of computational and algorithmic journalism (2013, background): “The possibilities of so-called “big data” can be glimpsed in the breadth of its potential application; not only have computational practices helped facilitate the growth of the academic subfield known as the digital humanities (Berry, 2011; Dalbello, 2011; Hayles, 2011; Rosenbloom, 2010; Slauter, 20”
- Diachronic word embeddings and semantic shifts: a survey (2018, background): “The release of the Google Books Ngrams corpus(1) played an important role in the development of the field and spurred work on the new discipline of ‘culturomics,’ studying human culture through digital media (Michel et al., 2011).”
- Statistically Significant Detection of Linguistic Change (2014, background): “It contains the frequency of short phrases of text (ngrams) that were extracted from books written in eight languages over five centuries [25].”