Zero-Shot Entity Linking by Reading Entity Descriptions

By Lajanugen Logeswaran, Ming-Wei Chang, Kenton C. T. Lee, Kristina Toutanova, Jacob Devlin, Honglak Lee. 2019

285 citations recorded by the source corpus.

We present the zero-shot entity linking task, where mentions must be linked to unseen entities without in-domain labeled data. The goal is to enable robust transfer to highly specialized domains, and so no metadata or alias tables are assumed. In this setting, entities are only identified by text descriptions, and models must rely strictly on language understanding to resolve the new entities. First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities. Second, we propose a simple and effective adaptive pre-training strategy, which we term domain-adaptive pre-training (DAP), to address the domain shift problem associated with linking unseen entities in a new domain. We present experiments on a new dataset that we construct for this task and show that DAP improves over strong pre-training baselines, including BERT. The data and code are available at https://github.com/lajanugen/zeshel.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Autoregressive Entity Retrieval (2020, background): “Third, computing an exact softmax over all entities is very expensive, hence current solutions need to subsample negative data (Logeswaran et al., 2019; Karpukhin et al., 2020) at training time.”
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020, methodology): “In the presence of domain shift between train and test data distributions of the same task, domain-adaptive pretraining (DAPT) is sometimes used to describe what we term TAPT (Logeswaran et al., 2019; Han and Eisenstein, 2019).”
Zero-shot Entity Linking with Dense Entity Retrieval (2019, methodology): “We also study zero-shot entity linking (Logeswaran et al., 2019).”
Relational World Knowledge Representation in Contextual Language Models: A Review (2021, methodology): “Logeswaran et al. (2019) propose “zero-shot entity linking,” whereby entity mention-spans must be”
Learning to Recognize Dialect Features (2020, methodology): “We also consider a novel multitask architecture, which learns across multiple features by encoding the feature names and descriptions, similar to recent work on few-shot or zero-shot multitask learning (Logeswaran et al., 2019; Brown et al., 2020).”
Zero-shot Learning by Generating Task-specific Adapters (2021, methodology): “Zero-shot setting has been explored in various NLP tasks, including text classiﬁcation (Yin et al., 2019), entity linking (Logeswaran et al., 2019) and entity typing (Obeidat et al., 2019).”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, background): “[337] explore BERT-based rankers with different architectures (e.”
Graph Retrieval-Augmented Generation: A Survey (2024, unknown): “This iterative process helps both components refine their performance in a coordinated manner.”
Learning to Generate Task-Specific Adapters from Task Description (2021, background): “Zeroshot learning (ZSL) has been explored for various NLP tasks, including text classification (Yin et al., 2019), entity linking (Logeswaran et al., 2019) and entity typing (Obeidat et al., 2019).”
A Survey of Model Architectures in Information Retrieval (2025, unknown): “The approaches for extracting query and document features have evolved from traditional term-based methods, such as boolean logic and vector space models, to modern solutions such as dense retrieval based on pre-trained language models (Lin et al., 2022).”