Multilingual Autoregressive Entity Linking

By Nicola De Cao, Ledell Yu Wu, Kashyap Popat, Mikel Artetxe, Naman Goyal, Mikhail Plekhanov, Luke S. Zettlemoyer, Nicola Cancedda, Sebastian Riedel, Fabio Petroni. 2021

100 citations recorded by the source corpus.

We present mGENRE, a sequence-to- sequence system for the Multilingual Entity Linking (MEL) problem—the task of resolving language-specific mentions to a multilingual Knowledge Base (KB). For a mention in a given language, mGENRE predicts the name of the target entity left-to-right, token-by-token in an autoregressive fashion. The autoregressive formulation allows us to effectively cross-encode mention string and entity names to capture more interactions than the standard dot product between mention and entity vectors. It also enables fast search within a large KB even for mentions that do not appear in mention tables and with no need for large-scale vector indices. While prior MEL works use a single representation for each entity, we match against entity names of as many languages as possible, which allows exploiting language connections between source input and target name. Moreover, in a zero-shot setting on languages with no training data at all, mGENRE treats the target language as a latent variable that is marginalized at prediction time. This leads to over 50% improvements in average accuracy. We show the efficacy of our approach through extensive evaluation including experiments on three popular MEL benchmarks where we establish new state-of-the-art results. Source code available at https://github.com/facebookresearch/GENRE.

Subfields: Artificial Intelligence, Management Science and Operations Research, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Information Extraction

Citations in the corpus

Editing Factual Knowledge in Language Models (2021, background): “Notably, language models that only rely on internal implicit memory are state-of-the-art for (multilingual-) Entity Linking (De Cao et al., 2021a,b).”
One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval (2021, methodology): “…models for many languages using pre-trained multilingual models such as mBERT or mT5 in many NLP tasks (e.g., entity linking: Botha et al., 2020; De Cao et al., 2021; semantic role labeling: Mulcaire et al., 2019b; Lyu et al., 2019; Fei et al., 2020; syntactic parsing: Mulcaire et al., 2019a;…”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “GENRE is extended to its multilingual version mGENRE [210].”
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation (2022, methodology): “have applied autoregressive generation models to conduct entity retrieval where queries are texts with a mention span and documents are entities in a knowledge base [5, 6].”
Highly Parallel Autoregressive Entity Linking with Discriminative Correction (2021, unknown): “For example, De Cao et al. (2021a) had to pre-train their model on Wikipedia abstracts.”