End-to-End Neural Entity Linking

By Nikolaos Kolitsas, Octavian-Eugen Ganea, Thomas Hofmann. 2018

288 citations recorded by the source corpus.

Entity Linking (EL) is an essential task for semantic text understanding and information extraction. Popular methods separately address the Mention Detection (MD) and Entity Disambiguation (ED) stages of EL, without leveraging their mutual dependency. We here propose the first neural end-to-end EL system that jointly discovers and links entities in a text document. The main idea is to consider all possible spans as potential mentions and learn contextual similarity scores over their entity candidates that are useful for both MD and ED decisions. Key components are context-aware mention embeddings, entity embeddings and a probabilistic mention - entity map, without demanding other engineered features. Empirically, we show that our end-to-end method significantly outperforms popular systems on the Gerbil platform when enough training data is available. Conversely, if testing datasets follow different annotation conventions compared to the training set (e.g. queries/ tweets vs news documents), our ED model coupled with a traditional NER system offers the best or second best EL accuracy.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Natural Language Processing

Citations in the corpus

Knowledge Enhanced Contextual Word Representations (2019, methodology): “1 Following Kolitsas et al. (2018), S e is used to score each of the candidate entities while incorporating the candidate entity prior from the KB.”
Autoregressive Entity Retrieval (2020, methodology): “Following Kolitsas et al. (2018), we considered only mentions that have entities in the KB (i.e., Wikipedia) and we used their candidate sets with the additions of the table computed by Hoffart et al. (2011).”
CM3: A Causal Masked Multimodal Model of the Internet (2022, background): “Speciﬁcally for the end-to-end Entity Linking, we aim to reproduce the setting of Kolitsas et al. (2018).”
Zero-shot Entity Linking with Dense Entity Retrieval (2019, background): “and Hofmann (2017), and Kolitsas et al. (2018) have established state-of-the-art results using neural networks to model context word, span and entity.”
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT (2019, background): “While this is based on the unrealistic assumption that we know the contents of the test set in advance, it is necessary for comparability with Peters et al. (2019), Kolitsas et al. (2018) and Broscheit (2019), who also design their entity vocabulary around the data.”
Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation (2019, background): “The insight behind performing schema linking is partly inspired by the success of incorporating entity linking in knowledge base question answering and semantic parsing (Yih et al., 2016; Krishnamurthy et al., 2017; Yu et al., 2018a; Herzig and Berant, 2018; Kolitsas et al., 2018).”
MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims (2019, methodology): “For this, we apply the state-of-the-art neural entity linking model by Kolitsas et al. (2018). This results in a total of 25,763 entities detected and linked to Wikipedia, with a total of 15,351 claims involved,”
Evaluating the Impact of Knowledge Graph Context on Entity Disambiguation Models (2020, background): “There is a wide variety of approaches in the literature for Entity Linking (EL) ranging from graph traversal [28], rule-based [23], to Neural Network-based approaches [7, 19].”
Self-Alignment Pretraining for Biomedical Entity Representations (2020, background): “Beyond the biomedical domain, entity linking has been intensively investigated on the less challenging general domain (Shen et al., 2015; Kolitsas et al., 2018).”
Joint Learning of Named Entity Recognition and Entity Linking (2019, background): “Kolitsas et al. (2018) proposed a model that performs mention detection instead of NER, not identifying the type of the detected mentions, as in our approach.”
Highly Parallel Autoregressive Entity Linking with Discriminative Correction (2021, unknown): “Entity Linking (EL; Bunescu and Pa¸sca, 2006; Cucerzan, 2007; Dredze et al., 2010; Hoffart et al., 2011; Le and Titov, 2018) is a fundamental task in NLP employed as a building block for text understanding (Févry et al., 2020a; Verga et al., 2020), question answering (Nie et al., 2019; Asai et al.,…”
Language Models are Open Knowledge Graphs (2020, background): “We will also investigate stronger entity linkers (Kolitsas et al., 2018) and learn a more robust relation mapping through weak or distant supervision (Mintz et al., 2009; Ratner et al., 2017).”
Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries (2020, background): “, 2011), neural EL (Francis-Landau et al., 2016; Kolitsas et al., 2018) Entity embeddings Text, entity mention spans Entity embeddings Joint embedding of entities and text (Yamada et al.”
Learning Cross-Context Entity Representations from Text (2020, background): “…and entity encoders in downstream entity linking systems (Sun et al., 2015; Yamada et al., 2016; 2017; Gupta et al., 2017; Murty et al., 2018; Kolitsas et al., 2018), but that previous work focuses solely on discriminating between entities that match a given mention according to an external…”
Relational World Knowledge Representation in Contextual Language Models: A Review (2021, background): “The minimally “entity-aware” BERT proposed by Broscheit (2019), which adds a single classiﬁcation layer on top of a pretrained BERT encoder, achieves competitive results with a state-of-the-art specialized entity linking architec-( ture (Kolitsas et al., 2018).”