Evaluation of a Cross-lingual Romanian-English Multi-document Summariser

By Constantin Orasan, Oana Andreea Chiorean. 2008

63 citations recorded by the source corpus.

The rapid growth of the Internet means that more information is available than ever before. Multilingual multi-document summarisation offers a way to access this information even when it is not in a language spoken by the reader by extracting the gist from related documents and translating it automatically. This paper presents an experiment in which Maximal Marginal Relevance (MMR), a well known multi-document summarisation method, is used to produce summaries from Romanian news articles. A task-based evaluation performed on both the original summaries and on their automatically translated versions reveals that they still contain a significant portion of the important information from the original texts. However, direct evaluation of the automatically translated summaries shows that they are not very legible and this can put off some readers who want to find out more about a topic.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization (2020, methodology): “(Lim et al., 2004; Orăsan and Chiorean, 2008; Wan et al., 2010) are widely used approaches in prior studies, they are prone to error propagation.”
Cross-language document summarization via extraction and ranking of multiple summaries (2019, background): “There are only a few pilot studies focusing on this task [16,19,22,23,27].”
NCLS: Neural Cross-Lingual Summarization (2019, methodology): “Orasan and Chiorean (2008) summarize the Romanian news with the maximal marginal relevance method (Goldstein et al., 2000) and produce the English summaries for English speakers.”
Phrase-based Compressive Cross-Language Summarization (2015, background): “Before Wan (2011)’s work that explicitly utilizes bilingual information in a graph-based framework, earlier methods often use information only from one language (de Chal-endar et al., 2005; Pingali et al., 2007; Orasan and Chiorean, 2008; Litvak et al., 2010).”
Cross-Language Document Summarization Based on Machine Translation Quality Prediction (2010, background): “Orasan and Chiorean (2008) propose to produce summaries with the MMR method from Romanian news articles and then automatically translate the summaries into English.”
A Robust Abstractive System for Cross-Lingual Summarization (2019, methodology): “’s (2017) pointer-generator network in PyTorch (Paszke et al., 2017). We pre-train for 12 epochs on the unmodified NYT corpus to obtain a baseline system. Table 2 shows the performance of this baseline on the unmodified NYT test set; our baseline underperforms the more complex systems of Paulus et a”
Global Voices: Crossing Borders in Automatic News Summarization (2019, background): “Previous work on cross-lingual summarization performs evaluation with human judgments (Orsan and Chiorean, 2008), or with automatic metrics and noisy source articles generated by automatic translation systems (Wan et al.”