Deep Joint Entity Disambiguation with Local Neural Attention

By Octavian-Eugen Ganea, Thomas Hofmann. 2017

358 citations recorded by the source corpus.

We propose a novel deep learning model for joint document-level entity disambiguation, which leverages learned neural representations. Key components are entity embeddings, a neural attention mechanism over local context windows, and a differentiable joint inference stage for disambiguation. Our approach thereby combines benefits of deep learning with more traditional approaches such as graphical models and probabilistic mention-entity maps. Extensive experiments show that we are able to obtain competitive or state-of-the-art accuracy at moderate computational costs.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Natural Language Processing

Citations in the corpus

Entities as Experts: Sparse Memory Access with Entity Supervision (2020, methodology): “, 2013); the Deep-Ed embeddings used by KNOWBERT (Ganea and Hofmann, 2017); or the random embeddings used by EAE.”
Learning Dense Representations for Entity Retrieval (2019, neutral): “Extensions include joint resolution over all entities in a document (Ratinov et al., 2011; Globerson et al., 2016; Ganea and Hofmann, 2017), joint modeling with related tasks like textual similarity (Yamada et al.”
Autoregressive Entity Retrieval (2020, methodology): “Following previous works (Yamada et al., 2016; Ganea & Hofmann, 2017; Le & Titov, 2018), we considered only mentions that have entities in the KB (i.e., Wikipedia).”
Zero-shot Entity Linking with Dense Entity Retrieval (2019, neutral): “5 Globerson et al. (2016)† 87.”
Scalable Zero-shot Entity Linking with Dense Entity Retrieval (2019, methodology): “To efficiently filter or rank the candidates, existing methods use different sources of external information, including manually curated mention tables (Ganea and Hofmann, 2017), incoming Wikipedia link popularity (Yamada et al.”
Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering (2019, neutral): “However, most entity linking systems (Ganea and Hofmann, 2017; Raiman and Raiman, 2018) have been trained on”
Multilingual Autoregressive Entity Linking (2021, positive): “Following previous work, we only evaluate in-KB links (Yamada et al., 2016; Ganea and Hofmann, 2017), i.e, we do not evaluate on mentions that link to entities out of the KB. Previous works considered Freebase (Bollacker et al., 2008) as KB, and thus we computed a mapping between Freebase ID and…”
Entity-aware ELMo: Learning Contextual Entity Representation for Entity Disambiguation (2019, methodology): “Yamada et al. (2016; 2017) and Ganea et al. (2017) learn entity representations by encoding the co-occurrence statistics between the entity and the words in its context.”
Mention and Entity Description Co-Attention for Entity Disambiguation (2018, methodology): “(Ganea and Hofmann 2017) and (Eshel, Cohen, and Radinsky 2017) apply the local attention of contexts side with neural network based methods.”
Zero-Shot Entity Linking by Reading Entity Descriptions (2019, methodology): “Mentions in context have been represented using e.g., CNN (Murty et al., 2018), LSTM (Gupta et al., 2017), or bag-of-word embeddings (Ganea and Hofmann, 2017).”
Improving Entity Linking by Modeling Latent Relations between Mentions (2018, methodology): “Ganea and Hofmann (2017) showed that such an approach can yield SOTA accuracy on a standard benchmark (AIDA-CoNLL dataset).”
Hyperbolic Entailment Cones for Learning Hierarchical Embeddings (2018, background): “…as link prediction in knowledge bases (Nickel et al., 2011; Bor-des et al., 2013) or in social networks (Hoff et al., 2002), text disambiguation (Ganea & Hofmann, 2017), word hyper-nymy (Shwartz et al., 2016), textual entailment (Rockt¨aschel et al., 2015) or taxonomy induction (Fu et al.,…”
Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation (2018, methodology): “For the future, the performance of our sys-tem can be easily improved combining it with a global method such as (Ganea and Hofmann, 2017).”
Attention, please! A survey of neural attention models in deep learning (2021, background): “(2017) and Ganea and Hofmann (2017) Entity description Ji et al.”
Joint Learning of Named Entity Recognition and Entity Linking (2019, methodology): “EL state-of-the-art methods often correspond to local methods which use as main features a candidate entity representation, a mention representation, and a representation of the mention’s context (Sun et al., 2015; Yamada et al., 2016, 2017; Ganea and Hofmann, 2017).”
REL: An Entity Linker Standing on the Shoulders of Giants (2020, methodology): “Ituses Wikipedia 2014 as the reference knowledge base and employs entity embed-dings provided by [8] for candidate selection.”
Learning Dynamic Context Augmentation for Global Entity Linking (2019, methodology): “Note that, our DCA model outperforms existing global models with the same local model (Global-RNN uses Berkeley-CNN as base model, Deep-ED and Ment-Norm use ETHZ-Attn as the local model).”
Joint Entity Linking with Deep Reinforcement Learning (2019, methodology): “Most of previous global models [14, 27, 37] calculate the pairwise scores between all candidate entities and select the most relevant group of entities.”