Highly Parallel Autoregressive Entity Linking with Discriminative Correction

By Nicola De Cao, Wilker Aziz, Ivan Titov. 2021

41 citations recorded by the source corpus.

Generative approaches have been recently shown to be effective for both Entity Disambiguation and Entity Linking (i.e., joint mention detection and disambiguation). However, the previously proposed autoregressive formulation for EL suffers from i) high computational cost due to a complex (deep) decoder, ii) non-parallelizable decoding that scales with the source sequence length, and iii) the need for training on a large amount of data. In this work, we propose a very efficient approach that parallelizes autoregressive linking across all potential mentions and relies on a shallow and efficient decoder. Moreover, we augment the generative objective with an extra discriminative component, i.e., a correction term which lets us directly optimize the generator’s ranking. When taken together, these techniques tackle all the above issues: our model is >70 times faster and more accurate than the previous generative method, outperforming state-of-the-art approaches on the standard English dataset AIDA-CoNLL. Source code available at https://github.com/nicola-decao/efficient-autoregressive-EL

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, methodology): “When it comes to downstream performances ( 2021a)) and we leave their application to SEAL as future work.”
Biases in Large Language Models: Origins, Inventory, and Discussion (2023, background): “The ield has witnessed a drastic change in paradigm with the advent and wide availability of large-scale pretrained language models, such as BERT [46], GPT [21, 108], T5 [109] and BART [79], which are now pervasive in every high-performance system for Machine Translation [25], Question Answering [95”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “Considering the efficiency challenges of generative EL approaches, [207] par-allelizes autoregressive linking across all potential mentions and relies on a shallow and efficient decoder.”