Lexicon Infused Phrase Embeddings for Named Entity Resolution

By Alexandre Passos, Vineet Kumar, Andrew McCallum. 2014

317 citations recorded by the source corpus.

Most state-of-the-art approaches for named-entity recognition (NER) use semi supervised information in the form of word clusters and lexicons. Recently neural network-based language models have been explored, as they as a byproduct generate highly informative vector representations for words, known as word embeddings. In this paper we present two contributions: a new form of learning word embeddings that can leverage information from relevant lexicons to improve the representations, and the first system to use neural word embeddings to achieve state-of-the-art results on named-entity recognition in both CoNLL and Ontonotes NER. Our system achieves an F1 score of 90.90 on the test set for CoNLL 2003---significantly better than any previous system trained on public data, and matching a system employing massive private industrial query-log data.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Natural Language Processing

In Information Extraction

Citations in the corpus

Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (2015, background): “Most, if not all, tasks in natural language processing can be cast as a question answering problem: high level tasks like machine translation (What is the translation into French?); sequence modeling tasks like named entity recognition [1] (NER) (What are the named entity tags in this sentence?) or ”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “[140] proposed to modify skip-gram models to better learn entity-type related word embeddings that can leverage information from relevant lexicons.”
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF (2016, background): “Most traditional high performance sequence labeling models are linear statistical models, including Hidden Markov Models (HMM) and Conditional Random Fields (CRF) (Ratinov and Roth, 2009; Passos et al., 2014; Luo et al., 2015), which rely heavily on hand-crafted features and taskspecific resources.”
Multilingual Language Processing From Bytes (2015, neutral): “Since 2003, better results have been reported using additional semi-supervised techniques (Ando and Zhang, 2005) and more recently, Passos et al. (2014) claimed the best English results (90.90% F1) using features derived from word-embeddings.”
Named Entity Recognition with Bidirectional LSTM-CNNs (2015, methodology): “Passos et al. (2014) obtained nearly the same performance using only public data by training phrase vectors in their lexicon-infused skip-gram model.”
What’s in an Embedding? Analyzing Word Embeddings through Multilingual Evaluation (2015, methodology): “They have been shown to be beneficial for several tasks such as machine translation (Botha and Blunsom, 2014), parsing (Lei et al., 2014), and named entity recognition (Passos et al., 2014).”
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer (2020, unknown): “…features, followed by feeding them to different linear classi-ﬁers such as maximum entropy, conditional random ﬁelds (CRFs), and support vector machines (Chieu and Ng, 2002; Florian et al., 2003; Finkel et al., 2005; Ratinov and Roth, 2009; Lin and Wu, 2009; Passos et al., 2014; Luo et al., 2015).”
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks (2016, unknown): “ 7 Published as a conference paper at ICLR 2017 Table 3: Comparison with state-of-the-art results (%). Model CoNLL 2000 CoNLL 2003 Spanish Dutch PTB 2003 Collobert et al. (2011) 94.32 89.59 – – 97.29 Passos et al. (2014) – 90.90 – – – Luo et al. (2015) – 91.2 – – – Huang et al. (2015) 94.46 90.10 – ”
Recent Trends in Deep Learning Based Natural Language Processing [Review Article] (2018, background): “[138] LExiCoN iNfuSED PHRASE EMBEDDiNgS 90.”
Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings (2015, background): “One approach is to utilize lexical embeddings to improve NER systems (Collobert and Weston, 2008; Turian et al., 2010; Passos et al., 2014), including for Twitter (Cherry and Guo, 2015).”
Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space (2014, methodology): “resentations using Canonical Correlation Analysis (CCA). Word vector representations or embeddings have been used in various NLP tasks such as named entity recognition (Neelakantan and Collins, 2014; Passos et al, 2014; Turian et al, 2010), dependency parsing (Bansal et al, 2014), chunking (Turian e”
A Joint Model for Entity Analysis: Coreference, Typing, and Linking (2014, methodology): “We use the same OntoNotes splits as for the coreference data; however, the New Testament ( NT ) (Ratinov and Roth, 2009) and the system of Passos et al. (2014).”