Contextual Dependencies in Unsupervised Word Segmentation

By Sharon Goldwater, T. Griffiths, Mark Johnson. Annual Meeting of the Association for Computational Linguistics, 2006

338 citations recorded by the source corpus.

Developing better methods for segmenting continuous text into words is important for improving the processing of Asian languages, and may shed light on how humans learn to segment speech. We propose two new Bayesian word segmentation methods that assume unigram and bigram models of word dependencies respectively. The bigram model greatly outperforms the unigram model (and previous probabilistic models), demonstrating the importance of such dependencies for word segmentation. We also show that previous probabilistic models rely crucially on sub-optimal search procedures.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/a99d6ebe6e583b752d1639a9002dd60581983dc1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Bayesian Nonparametrics: Hierarchical Bayesian nonparametric models with applications (2010, methodology): “When 0 < d < 1 the Pitman-Yor process yields power-law behavior (Pitman, 2002; Goldwater et al., 2006a; Teh, 2006a, see also Chapter ??).”
Unsupervised Pattern Discovery in Speech (2008, background): “More recently, Goldwater has improved upon these model-based approaches by allowing for sparse solutions and more thoroughly investigating the role of search in determining the optimal segmentation of the corpus [18].”
Adaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models (2006, methodology): “As a consequence, many nonparametric Bayesian models that have been used in comput ational linguistics, such as models of morphology [6] and word segmentation [7], can be expressed a ad ptor grammars.”
Inducing Tree-Substitution Grammars (2010, methodology): “The actual algorithm is analogous to the Gibbs sampler used for inference in the Bayesian model of word segmentation presented by Goldwater et al. (2006); indeed, the problem of inferring the derivations e from t can be viewed as a segmentation problem, where each full tree must be segmented into…”
A Bayesian framework for word segmentation: exploring the effects of context. (2009, positive): “…α1, a good balance is achieved between oversegmentation and undersegmentation of the corpus, and both token accuracy and lexicon 10Our implementation is slightly different than in the original presentation of this model (Goldwater et al., 2006a), and also fixes a small bug in that implementation.”
Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model (2018, methodology): “However, we employ recurrent neural networks at both steps, rather than n-gram backoff models (Goldwater et al., 2006; Andrews et al., 2017) or context-free models (Johnson et al.”
The Infinite PCFG Using Hierarchical Dirichlet Processes (2007, methodology): “DP mixture models have since been extended to hierarchical Dirichlet processes (HDPs) and HDP-HMMs (Teh et al., 2006; Beal et al., 2002) and applied to many different types of clustering/induction problems in NLP (Johnson et al., 2006; Goldwater et al., 2006).”
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP (2021, background): “17The idea and partial results are already presented in Goldwater et al. (2006a), but the authors request citing the updated 2009 paper.”
Reading to Learn: Constructing Features from Semantic Abstracts (2009, background): “1 There are many recent applications of Dirichlet processes in natural language processing, e.g. Goldwater et al. (2006).”