Zero-shot Entity Linking with Dense Entity Retrieval

By Ledell Yu Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, Luke S. Zettlemoyer. 2019

181 citations recorded by the source corpus.

We consider the zero-shot entity-linking challenge where each entity is defined by a short textual description, and the model must read these descriptions together with the mention context to make the final linking decisions. In this setting, retrieving entity candidates can be particularly challenging, since many of the common linking cues such as entity alias tables and link popularity are not available. In this paper, we introduce a simple and effective two-stage approach for zero-shot linking, based on fine-tuned BERT architectures. In the first stage, we do retrieval in a dense space defined by a bi-encoder that independently embeds the mention context and the entity descriptions. Each candidate is then examined more carefully with a cross-encoder, that concatenates the mention and entity text. Our approach achieves a nearly 6 point absolute gain on a recently introduced zero-shot entity linking benchmark, driven largely by improvements over previous IR-based candidate retrieval. We also show that it performs well in the non-zero-shot setting, obtaining the state-of-the-art result on TACKBP-2010. The code and pre-trained models are available at https://github.com/facebookresearch/BLINK.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Improving Wikipedia verifiability with AI (2022, methodology): “The dense retrieval sub-system is a neural network which learns from Wikipedia data to encode the citation context into a dense query vector (Wu et al., 2019; Karpukhin et al., 2020; Maillard et al., 2021; Oğuz et al., 2021; Luan et al., 2021).”
Autoregressive Entity Retrieval (2020, methodology): “As large generative models benefit from large amount of data, we first pre-train GENRE on the BLINK data (Wu et al., 2019), i.”
KILT: a Benchmark for Knowledge Intensive Language Tasks (2020, methodology): “For NQ, we additionally include gold passages among those to annotate, with the twofold objective of controlling the quality of the annotation process and filter out questions that can’t be an- 8https://yjernite.github.io/lfqa.html 9for Tf-idf and BLINK + flair we consider the first passage in the r”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, methodology): “This approach is widely used for retrieval (Lee et al., 2019; Reimers and Gurevych, 2019; Humeau et al., 2020; Wu et al., 2019).7 To learn lower-dimensional encodings, we learn down-projections from d = 768 to k ∈ 32, 64, 128, 512,8 implemented as a single feedforward layer, followed by layer…”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, background): “This case appears almost solved, confirming recent work demonstrating that dense retrieval is very effective at entity linking (Wu et al., 2019).”
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation (2019, methodology): “However, conventional KE models solely take the KG structures as input and do not involve textual data, and thus cannot directly help the pre-training of PLMs. Inspired by Xie et al. (2016), we take entity descriptions to bridge the gap between KE and PLM.”
Generating Fact Checking Briefs (2020, positive): “Entities in each claim are identiﬁed with BLINK (Wu et al., 2019), a model trained on Wikipedia data that links each entity to its nearest Wikipedia page.”
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “While, hard negatives are shown to be more important to train a dual-encoder (Gillick et al., 2019; Wu et al., 2019; Karpukhin et al., 2020; Luan et al., 2020; Xiong et al., 2020).”
♫ MuSiQue: Multihop Questions via Single-hop Question Composition (2021, methodology): “A SOTA wikification model (Wu et al., 2020) returns same result for a1 and e2 with the context of p1 + q1 and p2 + q2 respectively.”
Simple Entity-Centric Questions Challenge Dense Retrievers (2021, background): “Several recent works demonstrate re-trievers can easily learn dense representations for a large number of Wikipedia entities (Wu et al., 2020; Li et al., 2020), or directly generate entity names in an autoregressive manner (De Cao et al., 2021).”
Multilingual Autoregressive Entity Linking (2021, background): “Modern EL systems that use cross-encoding between context and entities usually do not score all entities in a KB as it is too computational expensive (Wu et al., 2020).”
Video Understanding as Machine Translation (2020, background): “Prior work in general-purpose encoder-based language models [9, 28], pre-trained with self-supervision on large textual corpora, led to strong improvements after finetuning on various downstream classification [53, 52] and ranking [58, 18] tasks.”