The American National Corpus: A Standardized Resource for American English

By Catherine Macleod, Nancy Ide, Ralph Grishman. 2000

108 citations recorded by the source corpus.

Linguistic research has become heavily reliant on text corpora over the past ten years. Such resources are becoming increasingly available through efforts such as the Linguistic Data Consortium (LDC) in the US and the European Language Resources Association (ELRA) in Europe. However, in the main the corpora that are gathered and distributed through these and other mechanisms consist of texts which can be easily acquired and are available for re-distribution without undue problems of copyright, etc. This practice has resulted in a vast over-representation among available corpora of certain genres, in particular newspaper samples, which comprise the greatest percentage of texts currently available from, for example, the LDC, and which also dominate the training data available for speech recognition purposes. Other available corpora typically consist of technical reports, transcriptions of parliamentary and other proceedings, short telephone conversations, and the like. The upshot of this is that corpusbased natural language processing has relied heavily on language samples representative of usage in a handful of limited and linguistically specialized domains.

Subfields: Artificial Intelligence, Language and Linguistics, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

In Evaluation & Benchmarks

Citations in the corpus

A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (2017, background): “t to be maximally diverse and roughly represent the full range of American English. We selected nine sources from the second release of the Open American National Corpus (OANC; Fillmore et al., 1998; Macleod et al., 2000; Ide and Macleod, 2001; Ide and Suderman, 2006, downloaded 12/20161), balancing”
The American National Corpus First Release (2004, background): “For additional information, see Ide & Macleod, 2001; Ide, et al., 2002; and Reppen & Ide, in press.”
Biases in Large Language Models: Origins, Inventory, and Discussion (2023, background): “While “balancing” has been the goal of the linguists behind the creation of historical corpora, such as the British National Corpus [21] and the American National Corpus [84], balancing larger corpora, such as those obtained from Common Crawl [27, 51], typically used to train large language models, ”
Integrating Linguistic Resources: The American National Corpus Model (2006, background): “The American National Corpus (ANC) project 1 (Ide and Macleod, 2001; Ide and Suderman, 2004) has released over 20 million words of spoken and written American English, available from the Linguistic Data Consortium.”