Advantages of query biased summaries in information retrieval

By Anastasios Tombros, Mark Sanderson. 1998

This paper presents an investigation into the utility of document summarisation in the context of information retrieval, more specifically in the application of so called query biased (or user directed) summaries: summaries customised to reflect the information need expressed in a query. Employed in the retrieved document list displayed after a retrieval took place, the summaries' utility was evaluated in a task-based environment by measuring users' speed and accuracy in identifying relevant documents. This was compared to the performance achieved when users were presented with the more typical output of an IR system: a static predefined summary composed of the title and first few sentences of retrieved documents. The results from the evaluation indicate that the use of query biased summaries significantly improves both the accuracy and speed of user relevance judgements.

Subfields: Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

Citations in the corpus

ELI5: Long Form Question Answering (2019, background): “Multi-document summarization The ELI5 task of writing a paragraph length response from multiple supporting documents can be seen as a form of query-based multi-document summarization (Tombros and Sanderson, 1998).”
Evaluation of Text Summarization in a Cross-lingual Information Retrieval Framework (2011, background): “Task-based evaluations (Mani et al., 1999a; Tombros et al., 1998; DUC2000, 2000) measure human performance using the summaries for a certain task (after the summaries are created).”
Exploratory Search: Beyond the Query-Response Paradigm (2009, background): “anchor text; or (4) to support document selection through query-biased summarization (Tombros and Sanderson, 1998).”
The Notion of Relevance in Information Science: Everybody knows what relevance is. But, what is it really? (2016, background): “Ø Tombros and Sanderson (1998) asked two groups of 10 graduate students each to judge the relevance of a list of the 50 highest ranked documents from 50 TREC 6.”
Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status (2002, background): “Extracts are often useful in an information retrieval environment since they give users an idea as to what the source document is about (Tombros and Sanderson 1998; Mani et al. 1999), but they are texts of relatively low quality.”
Find-similar: similarity browsing as a search tool (2006, unknown): “Our notion of query-biased similarity is more akin to querybiased summaries [36, 29] than to query-biased clustering”
User performance versus precision measures for simple search tasks (2006, methodology): “The summaries contained whole sentences from the original documents that contained some query words, with a bias towards the number of query words contained, and the proximity of the query terms within the sentence [16].”
Evaluating sentence-level relevance feedback for high-recall information retrieval (2018, background): “Tombros and Sanderson (1998) found that reviewers could identify more relevant documents for each query by reviewing the extracted summary, while, at the same time, making fewer labeling errors.”
SUMMAC: a text summarization evaluation (2002, background): “…and comprehension tasks (Morris, Kasper and Adams 1992), as well as tasks which measure the impact of summarization on determining the relevance of a document to a topic (Brandow, Mitze and Rau 1995; Mani and Bloedorn 1997; Jing, Barzilay, McKeown and El-hadad 1998; Tombros and Sanderson 1998).”
A Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate? (2005, result): “For example, in two previous studies (Mani, 2001; Tombros and Sanderson, 1998), users’ judgments were compared to “gold standard judgments” produced by members of the University of Pennsylvania’s Linguistic Data Consortium.”
A Proposal for Evaluating Answer Distillation from Web Data (2016, background): “A query-biased summarization task [17] involves a query-document-summary triple, where the system is given a query and a document, and produces a summary of the document that takes into account the query.”
Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022, background): “Query driven multi-document summarization (QFMD) aims to generate a summary according to the query and the provided relevant document(s) (Tombros and Sanderson, 1998).”