Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention

By Xiangyu Duan, Mingming Yin, Min Zhang, Boxing Chen, Weihua Luo. 2019

69 citations recorded by the source corpus.

Abstractive Sentence Summarization (ASSUM) targets at grasping the core idea of the source sentence and presenting it as the summary. It is extensively studied using statistical models or neural models based on the large-scale monolingual source-summary parallel corpus. But there is no cross-lingual parallel corpus, whose source sentence language is different to the summary language, to directly train a cross-lingual ASSUM system. We propose to solve this zero-shot problem by using resource-rich monolingual ASSUM system to teach zero-shot cross-lingual ASSUM system on both summary word generation and attention. This teaching process is along with a back-translation process which simulates source-summary pairs. Experiments on cross-lingual ASSUM task show that our proposed method is significantly better than pipeline baselines and previous works, and greatly enhances the cross-lingual performances closer to the monolingual performances.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Furu Wei
2. Li Dong
3. Pascale Fung
4. Mohit Bansal
5. Thomas Scialom
6. Wenhui Wang
7. Andrea Madotto
8. Jianfeng Gao

In Natural Language Processing

1. Furu Wei
2. Li Dong
3. Thomas Scialom
4. Jacopo Staiano
5. Paul-Alexis Dray
6. Sylvain Lamprier
7. Benjamin Piwowarski
8. Pascale Fung

Citations in the corpus

• WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization (2020, background): “Duan et al. (2019) propose a similar approach for cross-lingual abstractive sentence summarization.”
• MLSUM: The Multilingual Summarization Corpus (2020, background): “Most proposed approaches, so far, have used an extractive approach given the lack of a multilingual corpus to train abstractive models (Duan et al., 2019).”
• Cross-Lingual Natural Language Generation via Pre-Training (2019, background): “Another strand of work uses MT to generate pseudo training data for other language pairs that are lack of annotations (Shen et al. 2018; Duan et al. 2019).”
• Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning (2020, background): “, 2018), abstractive summarization (Duan et al., 2019; Zhu et al., 2019), and dependency parsing (Schuster”
• NCLS: Neural Cross-Lingual Summarization (2019, background): “Similarly, Duan et al. (2019) propose to use mono-lingual abstractive sentence summarization system to teach zero-shot cross-lingual abstractive sentence summarization on both summary word generation and attention.”