The use of MMR, diversity-based reranking for reordering documents and producing summaries

By Jaime Carbonell, Jade Goldstein-Stewart. 1998

This paper presents a method for combining query-relevance with information-novelty in the context of text retrieval and summarization. The Maximal Marginal Relevance (MMR) criterion strives to reduce redundancy while maintaining query relevance in re-ranking retrieved documents and in selecting appropriate passages for text summarization. Preliminary results indicate some benefits for MMR diversity ranking in document retrieval and in single document summarization. The latter are borne out by the recent results of the SUMMAC conference in the evaluation of summarization systems. However, the clearest advantage is demonstrated in constructing non-redundant multi-document summaries, where MMR results are clearly superior to non-MMR passage selection.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Determinantal Point Processes for Machine Learning (2012, background): “…places DPPs alongside a number of recently developed techniques for working with diverse sets, particularly in the information retrieval community [Carbonell and Goldstein, 1998, Zhai et al., 2003, Chen and Karger, 2006, Yue and Joachims, 2008, Radlinski et al., 2008, Swaminathan et al., 2009,…”
LexRank: Graph-based Lexical Centrality as Salience in Text Summarization (2004, methodology): “Several rerankers are implemented in MEAD, including one based on Maximal Marginal Relevance (MMR) (Carbonell & Goldstein, 1998) and the default reranker of the system based on Cross-Sentence Informational Subsumption (CSIS) (Radev, 2000).”
Neural Document Summarization by Jointly Learning to Score and Select Sentences (2018, methodology): “In recent years, extractive methods for summarization have proven effective in many systems (Carbonell and Goldstein, 1998; Mihalcea and Tarau, 2004; McDonald, 2007; Cao et al., 2015a).”
Abstractive Document Summarization with a Graph-Based Attentional Neural Model (2017, background): “Notable early works include (Edmundson, 1969; Carbonell and Goldstein, 1998; McDonald, 2007).”
Automatically Assessing Machine Summary Content Without a Gold Standard (2013, methodology): “Except for the CLASSY system, which performs more sophisticated redundancy removal, for the other methods we used the greedy Maximum Marginal Relevance technique (Carbonell and Goldstein 1998) for reducing redundancy.”
The Impact of Frequency on Summarization (2005, unknown): “The need for duplication removal and/or re-ranking was first discussed in (Carbonell and Goldstein, 1998), but the benefits and efficiency of such modules have never been studied rigorously.”
Large Language Models for Information Retrieval: A Survey (2023, unknown): “Additionally, reranking can accommodate other specific requirements, such as personalization [15–18] and diversification [19–22].”
Beyond independent relevance: methods and evaluation metrics for subtopic retrieval (2003, background): “We note that MinMix is most similar to the original MMR measure [2].”
BillSum: A Corpus for Automatic Summarization of US Legislation (2019, methodology): “To establish benchmarks on summarization performance, we evaluate several extractive summarization approaches by first scoring individual sentences, then using a selection strategy to pick the best subset (Carbonell and Goldstein, 1998).”
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data (2020, methodology): “Rather than conducting context aggregation of the retrieved articles, we consider an MMR summarizer (Carbonell and Goldstein, 1998) which performs extractive, multi-document summarization of text to generate a summaryAsumm (rightmost architecture in Fig.”
Information gain and divergence-based feature selection for machine learning-based text categorization (2006, unknown): “In document retrieval and summarization, marginal relevance should be maximized, hence the method is labeled as MMR (Carbonell & Goldstein, 1998).”
Learning to model relatedness for news recommendation (2011, background): “, [8, 39, 37, 3], often aim to locate relevant and novel information through finding a tradeoff between relevance and novelty.”
Improving Aggregate Recommendation Diversity Using Ranking-Based Techniques (2012, background): “, its importance has been discussed in information retrieval literature [35], [45], including some attempts to reduce redundancy and promote the diversity of retrieved results by reranking them [12], [38], [53].”
Introduction to the Special Issue on Summarization (2002, methodology): “This preprocessing yields a transcript on which standard techniques for extraction in text (here the use of MMR [Carbonell and Goldstein 1998] to identify relevant units) can operate successfully.”