XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages

By Tahmid Hasan, Abhik Bhattacharjee, Kazi Samin Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Rahman, Rifat Shahriyar. 2021

534 citations recorded by the source corpus.

Contemporary works on abstractive text summarization have focused primarily on high-resource languages like English, mostly due to the limited availability of datasets for low/mid-resource ones. In this work, we present XL-Sum, a comprehensive and diverse dataset comprising 1 million professionally annotated article-summary pairs from BBC, extracted using a set of carefully designed heuristics. The dataset covers 44 languages ranging from low to high-resource, for many of which no public dataset is currently available. XL-Sum is highly abstractive, concise, and of high quality, as indicated by human and intrinsic evaluation. We fine-tune mT5, a state-of-the-art pretrained multilingual model, with XL-Sum and experiment on multilingual and low-resource summarization tasks. XL-Sum induces competitive results compared to the ones obtained using similar monolingual datasets: we show higher than 11 ROUGE-2 scores on 10 languages we benchmark on, with some of them exceeding 15, as obtained by multilingual training. Additionally, training on low-resource languages individually also provides competitive performance. To the best of our knowledge, XL-Sum is the largest abstractive summarization dataset in terms of the number of samples collected from a single source and the number of languages covered. We are releasing our dataset and models to encourage future research on multilingual abstractive summarization. The resources can be found at \url{https://github.com/csebuetnlp/xl-sum}.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Additionally, only two of the 27 datasets were nonEnglish, despite much recent work that introduces multilingual summarization corpora (Giannakopoulos et al., 2015; Scialom et al., 2020; Ladhak et al., 2020; Hasan et al., 2021; Perez-Beltrachini and Lapata, 2021).”
PaLM 2 Technical Report (2023, background): “• XLSum (Hasan et al., 2021), which asks a model to summarize a news article in the same language in a single sentence, in Arabic, Bengali, English, Japanese, Indonesian, Swahili, Korean, Russian, Telugu, Thai, and Turkish.”
MEGA: Multilingual Evaluation of Generative AI (2023, methodology): “For a similar reason as the tagging datasets, we only evaluate on first 1000 examples of the test sets in different languages and recompute the baselines on the same testset using the weights of the XL-SUM pretrained model, opensourced by the authors (Hasan et al., 2021b).”
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022, methodology): “, 2019) and XLSum (Hasan et al., 2021)), and a class of data-to-text datasets with the communicative goal to generate game summaries from large sports statistic tables (e.”
Polyglot Prompt: Multilingual Multitask PrompTraining (2022, background): “5.2) et al., 2016) or multi-language learning in the same type of task, such as sequence labeling (Yang et al., 2016; Lin et al., 2018), text summarization (Hasan et al., 2021) and machine translation (Dong et al., 2015; Mahabadi et al., 2021; Hu et al., 2021).”
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning (2023, methodology): “As an ongoing research, we plan to extend the experiments to include more languages, tasks, models, criteria, and settings in future work to obtain broader and deeper insights.”
TaTa: A Multilingual Table-to-Text Dataset for African Languages (2022, background): “As noted in Section 2, others have mainly focused on summarization (Hasan et al., 2021; Varab and Schluter, 2021) and machine translation (Reid et al.”
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2024, background): “Previous works found that the Transformer possesses certain semantic capabilities in earlier layers Hasan et al. (2021).”