Global Voices: Crossing Borders in Automatic News Summarization
By Khanh Nguyen, Hal Daumé Iii. 2019
41 citations recorded by the source corpus.
We construct Global Voices, a multilingual dataset for evaluating cross-lingual summarization methods. We extract social-network descriptions of Global Voices news articles to cheaply collect evaluation data for into-English and from-English summarization in 15 languages. Especially, for the into-English summarization task, we crowd-source a high-quality evaluation dataset based on guidelines that emphasize accuracy, coverage, and understandability. To ensure the quality of this dataset, we collect human ratings to filter out bad summaries, and conduct a survey on humans, which shows that the remaining summaries are preferred over the social-network summaries. We study the effect of translation quality in cross-lingual summarization, comparing a translate-then-summarize approach with several baselines. Our results highlight the limitations of the ROUGE metric that are overlooked in monolingual summarization.
Subfields: Artificial Intelligence, Communication, Machine Translation, Summarization
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Rishi Bommasani
- Claire Cardie
- Yoav Goldberg
- Mohit Bansal
- Sebastian Riedel
- Yejin Choi
- Hal Daumé Iii
- Rada Mihalcea
In Machine Translation
In Summarization
Citations in the corpus
- Models and Datasets for Cross-Lingual Summarisation (2022, unknown): “…this reason, existing approaches either create large-scale synthetic data using back-translation (Zhu et al., 2019; Cao et al., 2020), translate the input documents (Ouyang et al., 2019), or build document-summary pairs from social media annotations and crowd-sourcing (Nguyen and Daumé III, 2019).”
- Intrinsic Evaluation of Summarization Datasets (2020, background): “Beyond English, both large summarization datasets (Nguyen and Daum ´ e III, 2019; Varab and Schluter, 2020) and more general language resources/technologies (Joshi et al., 2020) are less available, which may heighten the need for data quality assurance.”