From Baby Steps to Leapfrog: How “Less is More” in Unsupervised Dependency Parsing

By Valentin I. Spitkovsky, Hiyan Alshawi, Dan Jurafsky. 2010

232 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Understanding Dataset Difficulty with V-Usable Information (2021, neutral): “Text-based heuristics such as word identity (Bengio et al., 2009) or input length (Spitkovsky et al., 2010; Gururangan et al., 2018) have sometimes been used as proxies for instance difficulty, but offer limited insight into difficulty w.r.t. models.”
A Survey on Curriculum Learning (2021, background): “Difﬁculty Measurer * Angle Data Type ∝ Easy Sentence length [86], [107] Complexity Text - Number of objects [122] Complexity Images - # conj.”
Phrase Dependency Machine Translation with Quasi-Synchronous Tree-to-Tree Features (2014, background): “The NLP community has developed a range of statistical algorithms for building unsupervised parsers (Klein and Manning 2002, 2004; Smith 2006; Blunsom and Cohn 2010; Naseem et al. 2010; Spitkovsky, Alshawi, and Jurafsky 2010; Cohen 2011).”
A Multi-Pass Sieve for Coreference Resolution (2010, methodology): “…in many NLP applications: Brown et al. (1993) used simple models as “stepping stones” for more complex word alignment models; Collins (1999) used “cautious” decision list learning for named entity classiﬁcation; Spitkovsky et al. (2010) used “baby steps” for unsupervised dependency parsing, etc.”
Learning What Data to Learn (2017, background): “The measure of hardness in CL is typically determined by heuristic understandings of data (Bengio et al., 2009; Spitkovsky et al., 2010; Tsvetkov et al., 2016).”
A Competence-aware Curriculum for Visual Concepts Learning via Question Answering (2020, methodology): “The measures of hardness in curriculum learning approaches are usually determined by hand-designed heuristics [52,54,50,41].”
Curriculum-Based Meta-learning (2021, background): “For example, [4] takes the negative distance to the boundary to measure the hardness in classification, while [40] uses the sentence length as an indicator of hardness in studying grammar induction.”
Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking (2021, methodology): “We adopt a widely used strategy called baby step (Spitkovsky et al., 2010) to organize the scored data for CL. Speciﬁcally, we divide the score uniformly into N intervals and distribute the sorted data into N buckets accordingly.”
Information-Theoretic Measures of Dataset Difficulty (2021, methodology): “Heuristics such as vocabulary (Bengio et al., 2009) and input length (Spitkovsky et al., 2010; Gururangan et al., 2018) have sometimes been used as proxies for instance difﬁculty.”
Visually Grounded Neural Syntax Acquisition (2019, unknown): “, 2011) and dependency parsing (Klein and Manning, 2004; Smith and Eisner, 2006; Spitkovsky et al., 2010; Han et al., 2017).”
Inducing Tree-Substitution Grammars (2010, methodology): “, 2009), initialisation (Headden III et al., 2009; Spitkovsky et al., 2010), and extending the conditioning distributions (Headden III et al.”
Human-in-the-loop machine learning: a state of the art (2022, background): “Discrete schedulers adjust the training data subset after every fixed number ( > 1 ) of epochs or convergence on the current data subset and are commonly used due to their simplicity and effectiveness (Bengio et al. 2009; Spitkovsky et al. 2010; Ionescu et al. 2016).”
Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning (2016, background): “, initializing the learner with “simple” examples first, and then gradually increasing data complexity (Bengio et al., 2009; Spitkovsky et al., 2010).”
Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules (2013, unknown): “…used “cautious” decision list learning for named entity classiﬁcation; Borghesi and Favareto (1982) and Corazza et al. (1991) used “islands of reliability” approaches to parsing and speech recognition, and Spitkovsky et al. (2010) used “baby steps” for unsupervised dependency parsing, and so forth.”
An Empirical Exploration of Curriculum Learning for Neural Machine Translation (2018, unknown): “While organizing training samples based on difficulty has been demonstrated in NLP outside of neural models – e.g., Spitkovsky et al. (2010) bootstrap unsupervised dependency parsers by learning from incrementally longer sentences – curriculum learning has gained popularity to address the difficult…”
Word Representations: A Simple and General Method for Semi-Supervised Learning (2010, methodology): “For this reason, we hypothesize that learning representations over the most frequent words ﬁrst and gradually increasing the vocabulary—a curriculum training strategy (Elman, 1993; Bengio et al., 2009; Spitkovsky et al., 2010)—would provide better results than cleaning.”
Learning to Teach (2018, background): “The measure of hardness in curriculum learning (CL) (Bengio et al., 2009; Spitkovsky et al., 2010; Tsvetkov et al., 2016; Graves et al., 2017; Gong et al., 2016a) is typically determined by heuristic understandings of data.”