Scalable Zero-shot Entity Linking with Dense Entity Retrieval

By Ledell Yu Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, Luke S. Zettlemoyer. 2019

468 citations recorded by the source corpus.

This paper introduces a conceptually simple, scalable, and highly effective BERT-based entity linking model, along with an extensive evaluation of its accuracy-speed trade-off. We present a two-stage zero-shot linking algorithm, where each entity is defined only by a short textual description. The first stage does retrieval in a dense space defined by a bi-encoder that independently embeds the mention context and the entity descriptions. Each candidate is then re-ranked with a cross-encoder, that concatenates the mention and entity text. Experiments demonstrate that this approach is state of the art on recent zero-shot benchmarks (6 point absolute gains) and also on more established non-zero-shot evaluations (e.g. TACKBP-2010), despite its relative simplicity (e.g. no explicit entity embeddings or manually engineered mention tables). We also show that bi-encoder linking is very fast with nearest neighbor search (e.g. linking with 5.9 million candidates in 2 milliseconds), and that much of the accuracy gain from the more expensive cross-encoder can be transferred to the bi-encoder via knowledge distillation. Our code and models are available at https://github.com/facebookresearch/BLINK.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Re2G: Retrieve, Rerank, Generate (2022, methodology): “GENRE [Cao et al., 2021] is trained on BLINK [Wu et al., 2020] and all KILT tasks jointly using a sequence-to-sequence language model to generate the title of the Wikipedia page where the answer can be found.”
Improving Wikipedia verifiability with AI (2022, methodology): “We use Sphere (Piktus et al., 2021), a web-scale corpus and search infrastructure for web-scale data, as a source of candidate web pages.”
Multi-Task Retrieval for Knowledge-Intensive Tasks (2021, background): “Finally two entity linkers, GENRE (Cao et al., 2020) and BLINK (Wu et al., 2020), are worth mentioning.”
Robust Retrieval Augmented Generation for Zero-shot Slot Filling (2021, methodology): “It is trained on BLINK (Wu et al., 2020) and all KILT tasks jointly.”
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2020, methodology): “Hence, it is likely to bring noise if we simply sample hard negatives from the top-retrieved passages by the dense retriever, which is a widely adopted strategy to sample hard negatives in previous studies (Gillick et al., 2019; Wu et al., 2020; Xiong et al., 2020).”
ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering (2021, methodology): “Schema Retriever As schema items are not always mentioned explicitly in the question and their vocabularies are much fewer than entities 13 , we leverage the dense retriever framework (Mazar´e et al., 2018; Humeau et al., 2020; Wu et al., 2020) to obtain the related types and relations.”
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023, background): “…Gu et al., 2020; Peng et al., 2019), NER across more datasets (Tjong Kim Sang and De Meulder, 2003; Malmasi et al., 2022; Fu et al., 2022; Laskar et al., 2022a (Wu et al., 2020; Ayoola et al., 2022; Laskar et al., 2022b,c), Answer Re-Ranking Task (Garg et al., 2020; Laskar et al., 2020b), etc.”
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval (2021, background): “…distance ( i.e., dissimilarity) for visualization: the longer the distance is, the less similar it is. et al., 2020b), information retrieval (Luan et al., 2021; Khattab and Zaharia, 2020), dialogue (Ji et al., 2014; Henderson et al., 2017) and entity linking (Gillick et al., 2019; Wu et al., 2020).”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, methodology): “As one of the most essential techniques to overcome information overload, text retrieval systems have been widely employed to support many downstream applications, including question answering [1], [2], dialog system [3], [4], entity linking [5], [6] and Web search [7].”
Phrase Retrieval Learns Passage Retrieval, Too (2021, methodology): “…to outperform traditional sparse retrieval methods such as TF-IDF and BM25 in a range of knowledge-intensive NLP tasks (Petroni et al., 2021), including open-domain question answering (QA) (Chen et al., 2017), entity linking (Wu et al., 2020), and knowledge-grounded dialogue (Dinan et al., 2019).”
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering (2020, background): “Similar focused cross-attention methods have recently proved to be effective for Entity Linking (Wu et al., 2020) tasks, although they annotate the inputs instead of the retrieved passages.”
Relational World Knowledge Representation in Contextual Language Models: A Review (2021, methodology): “Here, competitive solutions train separate BERT models to select and rank candidate entities by encoding their descriptions (Logeswaran et al., 2019; Wu et al., 2020).”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models (2024, unknown): “In this part, we will briefly review studies with these two types of generators: parameter-accessible (white-box) and parameter-inaccessible (black-box).”
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021, background): “Such a two-stage procedure is particularly useful in a variety of natural language processing tasks, including question answering (Mao et al., 2021; Xiong et al., 2020b), dialogue system (Ji et al., 2014; Henderson et al., 2017) and entity linking (Gillick et al., 2019; Wu et al., 2020).”
Highly Parallel Autoregressive Entity Linking with Discriminative Correction (2021, unknown): “This is in line with many previous models (Luo (Hoffart et al., 2011) dataset. et al., 2015; Ganea and Hofmann, 2017; Yamada et al., 2016) and all systems we compare to.”