Class-Based n-gram Models of Natural Language

By Peter F. Brown, V. D. Pietra, P. D. Souza, Jennifer C. Lai, Robert L. Mercer. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019, background): “Learning widely applicable representations of words has been an active area of research for decades, including non-neural (Brown et al., 1992; 1Will be released before the end of October 2018.”
Recurrent Neural Network Grammars (2016, methodology): “To obtain clusters, we use the greedy agglomerative clustering algorithm of Brown et al. (1992).”
One billion word benchmark for measuring progress in statistical language modeling (2013, neutral): “…mention some of the most popular ones: • N-grams with Modified Kneser-Ney smoothing (Chen and Goodman, 1996) • Cache (Jelinek et al., 1991) • Class-based (Brown et al., 1992) • Maximum entropy (Rosenfeld, 1994) • Structured (Chelba and Jelinek, 2000) • Neural net based (Bengio et al., 2003) •…”
Experience Grounds Language (2020, background): “Local contexts proved powerful when combined with agglomerative clustering guided by mutual information (Brown et al., 1992).”
MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, background): “…mainly leverage pre-training to provide language representations and features to the downstream tasks, which includes word-level representations (Brown et al., 1992; Ando & Zhang, 2005; Blitzer et al., 2006; Collobert & Weston, 2008; Mikolov et al., 2013; Pennington et al., 2014) and…”
Parameter-Efficient Transfer Learning for NLP (2019, background): “Brown clusters, trained on distributional information, are a classic example of pre-trained representations (Brown et al., 1992).”
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark (2021, background): “…Learning Semi-supervised and transfer learning have grown into cornerstones of NLP. Early work learned unsupervised representations of words (Brown et al. 1992; Mikolov et al. 2013), while more recent work employs contextualized representations from neural language models (Peters et al.…”
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter (2016, methodology): “Brown Clusters and Length We highlight the use of Brown Clusters (Brown et al., 1992) and length features (as inspired by Nobata et al. (2016)), as these are the only two features that classify the minority classes for both amateur and expert annotators.”
A Survey on Hate Speech Detection using Natural Language Processing (2017, methodology): “A standard al-gorithm for this is Brown clustering (Brown et al., 1992) which has been used as a feature in Warner and Hirschberg (2012).”
A bit of progress in language modeling (2001, methodology): “We also use computational tricks adapted from Brown et al. (1992).”
Classes for fast maximum entropy training (2001, background): “Word classes have, of course, been used extensively in language modeling, including [2][3][4][5].”
Data Noising as Smoothing in Neural Network Language Models (2017, neutral): “However, it forms what Brown et al. (1992) term a “sticky pair”: the unigram “Dumpty” almost always follows the unigram “Humpty”, and similarly, “Humpty” almost always precedes “Dumpty”.”
Transductive Multi-View Zero-Shot Learning (2015, methodology): “, Wikipedia by natural language processing models such as [4], [32].”
Joint Language and Translation Modeling with Recurrent Neural Networks (2013, methodology): “We obtain word-classes using a version of Brown-Clustering with an additional regularization term to optimize the runtime of the language model (Brown et al., 1992; Zweig and Makarychev, 2013).”
Domain Adaptation with Structural Correspondence Learning (2006, background): “There are many choices for modeling co-occurrence data (Brown et al., 1992; Pereira et al., 1993; Blei et al., 2003).”
Dependency-Based Word Embeddings (2014, background): “On one end of the spectrum, words are grouped into clusters based on their contexts (Brown et al., 1992; Uszkoreit and Brants, 2008).”