Leveraging Passage-level Cumulative Gain for Document Ranking

By Zhijing Wu, Jiaxin Mao, Yiqun Liu, Jingtao Zhan, Yukun Zheng, Min Zhang, Shaoping Ma. 2020

40 citations recorded by the source corpus.

Document ranking is one of the most studied but challenging problems in information retrieval (IR) research. A number of existing document ranking models capture relevance signals at the whole document level. Recently, more and more research has begun to address this problem from fine-grained document modeling. Several works leveraged fine-grained passage-level relevance signals in ranking models. However, most of these works focus on context-independent passage-level relevance signals and ignore the context information, which may lead to inaccurate estimation of passage-level relevance. In this paper, we investigate how information gain accumulates with passages when users sequentially read a document. We propose the context-aware Passage-level Cumulative Gain (PCG), which aggregates relevance scores of passages and avoids the need to formally split a document into independent passages. Next, we incorporate the patterns of PCG into a BERT-based sequential model called Passage-level Cumulative Gain Model (PCGM) to predict the PCG sequence. Finally, we apply PCGM to the document ranking task. Experimental results on two public ad hoc retrieval benchmark datasets show that PCGM outperforms most existing ranking models and also indicates the effectiveness of PCG signals. We believe that this work contributes to improving ranking performance and providing more explainability for document ranking.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jimmy J. Lin
2. Andrew Yates
3. Rodrigo Nogueira
4. Kai Hui
5. Ben He
6. Xianpei Han
7. Le Sun
8. Zhi Zheng

In Information Systems

1. Rodrigo Nogueira
2. Andrew Yates
3. Jimmy J. Lin
4. Kai Hui
5. Ben He
6. Le Sun
7. Zhi Zheng
8. Xianpei Han

In Information Retrieval & Search

1. Jimmy J. Lin
2. Rodrigo Nogueira
3. Andrew Yates
4. Kai Hui
5. Ben He
6. Le Sun
7. Xianpei Han
8. Zhi Zheng

Citations in the corpus

• Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “limitation is addressed by a number of models [1, 4, 20, 25, 30, 39], and a simple retrieve-then-rerank approach can be elaborated into”
• BERT-QE: Contextualized Query Expansion for Document Re-ranking (2020, methodology): “Wu et al. (Wu et al., 2020) propose the context-aware Passage-level Cumulative Gain to aggregate relevance scores of passages, which is incorporated into a BERT-based model for document ranking.”