One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval

By Akari Asai, Xinyan Velocity Yu, Jungo Kasai, Hannaneh Hajishirzi. 2021

83 citations recorded by the source corpus.

We present Cross-lingual Open-Retrieval Answer Generation (CORA), the first unified many-to-many question answering (QA) model that can answer questions across many languages, even for ones without language-specific annotated data or knowledge sources. We introduce a new dense passage retrieval algorithm that is trained to retrieve documents across languages for a question. Combined with a multilingual autoregressive generation model, CORA answers directly in the target language without any translation or in-language retrieval modules as used in prior work. We propose an iterative training method that automatically extends annotated data available only in high-resource languages to low-resource ones. Our results show that CORA substantially outperforms the previous state of the art on multilingual open QA benchmarks across 26 languages, 9 of which are unseen during training. Our analyses show the significance of cross-lingual retrieval and generation in many languages, particularly under low-resource settings.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hannaneh Hajishirzi
2. Luke S. Zettlemoyer
3. Sebastian Riedel
4. Danqi Chen
5. Wen-tau Yih
6. Mohit Bansal
7. Jianfeng Gao
8. Hinrich Schütze

In Information Systems

1. Wayne Xin Zhao
2. Ji-rong Wen
3. Sewon Min
4. Wen-tau Yih
5. Hannaneh Hajishirzi
6. Akari Asai
7. Danqi Chen
8. Jing Liu

In Information Retrieval & Search

1. Sebastian Riedel
2. Wen-tau Yih
3. Hannaneh Hajishirzi
4. Danqi Chen
5. Gautier Izacard
6. Akari Asai
7. Edouard Grave
8. Patrick Lewis

In Natural Language Processing

1. Hannaneh Hajishirzi
2. Sebastian Riedel
3. Luke S. Zettlemoyer
4. Wen-tau Yih
5. Danqi Chen
6. Mohit Bansal
7. Akari Asai
8. Edouard Grave

Citations in the corpus

• Augmented Language Models: a Survey (2023, background): “In contrast, dense neural retrievers use a dense query and dense document vectors obtained from a neural network (Asai et al., 2021).”
• Task-aware Retrieval with Instructions (2022, methodology): “, BM25; Robertson and Zaragoza 2009) across domains when training data is abundant (Luo et al., 2022; Asai et al., 2021; Petroni et al., 2021).”
• Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, unknown): “Concretely, retrieved assets not only impose an additional velocity on particles for distribution initialization, but also help optimize the 2D diffusion model through low-rank adaptation.”
• Retrieval-based Language Models and Applications (2023, methodology): “At this end, we will cover recent extensions of retrieval-based LMs beyond English text, including multilingual (Asai et al., 2021), multi-modal (Chen et al., 2022; Yasunaga et al., 2022) and code (Parvez et al., 2021) retrieval.”
• Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, background): “[322] further propose a multilingual dense passage retriever that extends DPR to retrieve candidate passages from multilingual document collections in a cross-lingual way.”
• HyQE: Ranking Contexts with Hypothetical Query Embeddings (2024, methodology): “Recent advancements in embedding models such as BERT (Kenton and Toutanova, 2019; Reimers and Gurevych, 2019) allow capturing text semantics through dense vector representations (Asai et al., 2021).”
• BERGEN: A Benchmarking Library for Retrieval-Augmented Generation (2024, unknown): “Following [4], we use MKQA [55] and XOR-TyDi QA [2] datasets for multilingual evaluation in our experiments.”