Evaluating Entity Linking with Wikipedia

By Ben Hachey, Will Radford, Joel Nothman, Matthew Honnibal, James Richard Curran. 2013

291 citations recorded by the source corpus.

Named Entity Linking (nel) grounds entity mentions to their corresponding node in a Knowledge Base (kb). Recently, a number of systems have been proposed for linking entity mentions in text to Wikipedia pages. Such systems typically search for candidate entities and then disambiguate them, returning either the best candidate or nil. However, comparison has focused on disambiguation accuracy, making it difficult to determine how search impacts performance. Furthermore, important approaches from the literature have not been systematically compared on standard data sets. We reimplement three seminal nel systems and present a detailed evaluation of search strategies. Our experiments find that coreference and acronym handling lead to substantial improvement, and search strategies account for much of the variation between systems. This is an interesting finding, because these aspects of the problem have often been neglected in the literature, which has focused largely on complex candidate ranking algorithms.

Subfields: Artificial Intelligence, Management Science and Operations Research, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Learning Dense Representations for Entity Retrieval (2019, background): “These variations and the resulting fragmentation of evaluation is discussed at length by Ling et al. (2015) and Hachey et al. (2013), and partially addressed by attempts to consolidate datasets (Cornolti et al., 2013) and metrics (Usbeck et al., 2015).”
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking (2019, background): “Hachey et al. (2013); Guo et al. (2013); Durrett and Klein (2014) showed the inﬂu-ence of CG on entity linking, because it can be the coverage bottleneck, when the correct entity is not contained in the candidates for ED.”
Learning Text Representations for 500K Classification Tasks on Named Entity Disambiguation (2018, background): “Hachey et al. (2012) present a detailed overview of all possible components, but in this section we will focus on the most relevant high performing systems.”
Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions (2015, background): “WSD is the task to identify the sense of a word (rather than a named entity) in the context from a sense inventory (e.g., WordNet [32]) instead of a knowledge base.”
A Joint Model for Entity Analysis: Coreference, Typing, and Linking (2014, background): “As noted by Hachey et al. (2013) and Guo et al. (2013), candidate generation is often overlooked and yet accounts for large gaps in performance between different systems.”
Knowledge vault: a web-scale approach to probabilistic knowledge fusion (2014, methodology): “The in-house named entity linkage system we use is similar to the methods described in [17].”
ProjE: Embedding Projection for Knowledge Graph Completion (2016, background): “Knowledge Graphs (KGs) have become a crucial resource for many tasks in machine learning, data mining, and artificial intelligence applications including question answering [34], entity disambiguation [7], named entity linking [14], fact checking [32], and link prediction [28] to name a few.”
WikiRef: Wikilinks as a route to recommending appropriate references for scientific Wikipedia pages (2018, methodology): “Further, Wikipedia is being widely utilized by many applications focused on entity linking and disambiguation (Moro et al., 2014; Hachey et al., 2013; Cucerzan, 2007; Hoffart et al., 2011), named entity disambiguation (Cucerzan, 2007; Barrena et al.”
Joint Coreference Resolution and Named-Entity Linking with Multi-Pass Sieves (2013, background): “4 Haghighi and Klein (2009) 77.”
Entity Disambiguation with Web Links (2015, methodology): “Analysis shows that these can have a large effect on system performance (Hachey et al., 2013; Piccinno and Ferragina, 2014).”
Collective Entity Resolution with Multi-Focal Attention (2016, methodology): “Most ER systems consist of a mention model , a context model , and a coherence model (Milne and Witten, 2008; Cucerzan, 2007; Ratinov et al., 2011; Hoffart et al., 2011; Hachey et al., 2013).”