LETOR: A benchmark collection for research on learning to rank for information retrieval

By Tao Qin, Tie-Yan Liu, Jun Xu, Hang Li. Information retrieval (Boston), 2010

526 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/7137a80d199e82397a564696e12773256f343d17

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiafeng Guo
2. Xueqi Cheng
3. Yixing Fan
4. Jun Xu
5. W. Bruce Croft
6. Yanyan Lan
7. Andrew Yates
8. Yi Chang

In Information Systems

1. Jiafeng Guo
2. Yixing Fan
3. Jun Xu
4. Xueqi Cheng
5. Yi Chang
6. Yanyan Lan
7. Chengxiang Zhai
8. Olivier Chapelle

Citations in the corpus

• Multivariate spearman's ρ for aggregating ranks using copulas (2016, methodology): “Indeed, we also perform better than Qin et al. (2010a) where the best result uses a coset-permutation distance based stagewise (CPS) model with Spearman’s ρ in a probabilistic model.”
• Information Retrieval: A Biomedical and Health Perspective (2020, background): “Maslove raises several concerns, such as biomedical research being different from fields like physics, where the larger public may see and take action on papers they find that have not been subject to peer review [198].”
• A cross-benchmark comparison of 87 learning to rank methods (2015, background): “Well-known benchmark datasets in the learning to rank field include the Yahoo! Learning to Rank Challenge datasets (Chapelle and Chang, 2011), the Yandex Internet Mathematics 2009 contest, the LETOR datasets (Qin et al., 2010b), and the MSLR (Microsoft Learning to Rank) datasets.”
• Regret Lower Bound and Optimal Algorithm in Dueling Bandit Problem (2015, methodology): “MSLR: We tested submatrices of a136× 136 preference matrix fromZoghi et al.(2015), which is derived from the Microsoft Learning to Rank (MSLR) dataset (Microsoft Research, 2010; Qin et al., 2010) that consists of relevance information between queries and documents with more than30K queries.”
• Yahoo! Learning to Rank Challenge Overview (2010, background): “The release of the letor benchmark (Qin et al., 2010)2 in 2007 was a formidable boost for the development of learning to rank algorithms because researchers were able for the first time to compare their algorithms on the same benchmark datasets.”
• Learning to Rank for Information Retrieval and Natural Language Processing (2011, unknown): “NDCG • Imperfect ranking decreases NDCG MAP • Evaluating ranking using two grades • AP MAP (cont’) • Example: perfect ranking – (1,0,1,1,0,0,0) grade r=0,1 – (1, -, 0.67, 0.75, -, -, -) P(j) precision at position j – 0.81 AP average precision Relations with Other Learning Tasks • No need to predict ”
• Modeling Diverse Relevance Patterns in Ad-hoc Retrieval (2018, methodology): “To evaluate the performance of our model, we conducted experiments on two LETOR benchmark datasets [28]: Million Query Track 2007 (MQ2007) and Million Query Track 2008 (MQ2008).”