Phrase Clustering for Discriminative Learning

By Dekang Lin, Xiaoyun Wu. 2009

253 citations recorded by the source corpus.

We present a simple and scalable algorithm for clustering tens of millions of phrases and use the resulting clusters as features in discriminative classifiers. To demonstrate the power and generality of this approach, we apply the method in two very different applications: named entity recognition and query classification. Our results show that phrase clusters offer significant improvements over word clusters. Our NER system achieves the best current result on the widely used CoNLL benchmark. Our query classifier is on par with the best system in KDDCUP 2005 without resorting to labor intensive knowledge engineering efforts.

Subfields: Artificial Intelligence, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Extraction

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, background): “Other related approaches exist, like phrase clustering (Lin and Wu, 2009) which has been shown to work well for NER.”
Neural Architectures for Named Entity Recognition (2016, methodology): “Our models obtain state-of-the-art performance in NER in four languages without resorting to any language-specific knowledge or resources such as gazetteers.”
Lexicon Infused Phrase Embeddings for Named Entity Resolution (2014, background): “This result matches the highest F1 previously reported, in Lin and Wu (2009), but is the ﬁrst system to do so without using massive private data.”
Joint Entity Recognition and Disambiguation (2015, background): “Lin and Wu (2009) cluster tens of millions of phrases and use the resulting clusters as features in NER reporting the best performance on the CoNLL’03 English NER data set.”
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer (2020, unknown): “…features, followed by feeding them to different linear classi-ﬁers such as maximum entropy, conditional random ﬁelds (CRFs), and support vector machines (Chieu and Ng, 2002; Florian et al., 2003; Finkel et al., 2005; Ratinov and Roth, 2009; Lin and Wu, 2009; Passos et al., 2014; Luo et al., 2015).”
Word Representations: A Simple and General Method for Semi-Supervised Learning (2010, background): “Lin and Wu (2009) ﬁnds that the representations that are good for NER are poor for search query classiﬁcation, and vice-versa.”
Visual Attention Model for Name Tagging in Multimodal Social Media (2018, background): “…2016; Ma and Hovy, 2016) proposed several neural network architectures for named tagging that outperform traditional explicit features based methods (Chieu and Ng, 2002; Florian et al., 2003; Ando and Zhang, 2005; Ratinov and Roth, 2009; Lin and Wu, 2009; Passos et al., 2014; Luo et al., 2015).”
Domain-specific Knowledge Graphs: A survey (2020, methodology): “ction (IE) techniques and sophisticated Natural Language Processing (NLP) approaches. Examples of techniques used for entities recognition and relations extraction are; Conditional Random Field (CRF) [20], machine learning models (e.g. SVM), neural network models such as Bidirectional Long Short-Ter”
Recovering Semantics of Tables on the Web (2011, methodology): “Existing methods for extracting classes of instances from the Web require sets of instances that are each either unlabeled [18, 23, 29], or associated with a class label [1, 13, 21, 22, 30].”
Semi-supervised Relation Extraction with Large-scale Word Clustering (2011, background): “Moreover, extending word clustering to phrase clustering (Lin and Wu, 2009) and pattern clustering (Sun and Grishman, 2010) is worth future investigation for relation extraction.”
Tailoring Continuous Word Representations for Dependency Parsing (2014, methodology): “We tried two kinds of indicator features: Bucket features: For both parent and child vectors in a potential dependency, we fire one indicator feature per dimension of each embedding 4A similar effect, when changing distributional context window sizes, was found by Lin and Wu (2009).”
Coreference Semantics from Web Features (2012, methodology): “We check the match of not just the top 1 cluster ids, but also farther down in the 20 sized lists because, as discussed in Lin and Wu (2009), the soft cluster assignments often reveal different senses of a word.”
Few-Shot Named Entity Recognition: A Comprehensive Study (2020, unknown): “Early studies have noticed the importance of reducing the annotation labor, where semi-supervised learning is employed, such as clustering (Lin and Wu, 2009), and combining supervised objective with unsupervised word representations (Turian et al.”
Handbook of Big Data (2016, methodology): “1 k-Means Clustering-Based Model Reduction k-means clustering is a standard clustering technique [14] and is widely used in many applications [8,10].”
Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition (2011, background): “Lin and Wu (2009) report an F 1 score of 90.90 on the original split of the CoNLL data.”
Toward Mention Detection Robustness with Recurrent Neural Networks (2016, background): “…Kim Sang and De Meulder, 2003; Florian et al., 2003; Miller et al., 2004; Ando and Zhang, 2005; Suzuki and Isozaki, 2008; Ratinov and Roth, 2009; Lin and Wu, 2009; Turian et al., 2010; Ritter et al., 2011; Passos et al., 2014; Cherry and Guo, 2015) and mention detection (Florian et al., 2004)…”
N-gram-Based Low-Dimensional Representation for Document Classification (2014, methodology): “N -gram features are not commonly used in text classification, probably because the dictionary Dn tends to grow exponentially with n. Phrase structure extraction can be used to identify only n-grams which are phrase patterns, and thus limit the dictionary size.”