A Study of MatchPyramid Models on Ad-hoc Retrieval

By Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Xueqi Cheng. 2016

90 citations recorded by the source corpus.

Deep neural networks have been successfully applied to many text matching tasks, such as paraphrase identification, question answering, and machine translation. Although ad-hoc retrieval can also be formalized as a text matching task, few deep models have been tested on it. In this paper, we study a state-of-the-art deep matching model, namely MatchPyramid, on the ad-hoc retrieval task. The MatchPyramid model employs a convolutional neural network over the interactions between query and document to produce the matching score. We conducted extensive experiments to study the impact of different pooling sizes, interaction functions and kernel sizes on the retrieval performance. Finally, we show that the MatchPyramid models can significantly outperform several recently introduced deep matching models on the retrieval task, but still cannot compete with the traditional retrieval models, such as BM25 and language models.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Information Systems

In Information Retrieval & Search

Citations in the corpus

Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “es explicit histograms, while KNRM uses Gaussian kernels to create differentiable “soft histograms” that allow the embeddings to be ﬁne-tuned during training. Position-aware models like MatchPyramid [Pang et al., 2016], PACRR [Hui et al., 2017], Co-PACRR [Hui et al., 2018], and ConvKNRM [Dai et al.,”
Learning to Match using Local and Distributed Representations of Text for Web Search (2016, methodology): “While some negative results have been reported for title-based DSSM and CDSSM on the ad hoc document retrieval tasks [12, 30], we included document-based variants appropriately retrained on the same set of positive query and document pairs as our model.”
Neural information retrieval: at the end of the early years (2017, methodology): “…made in computer vision through employing a very large number of hidden layers (hence ‘‘deep’’ learning), such deep structures have typically been less effective in NLP and IR than more shallow architectures (Pang et al. 2016a), though again with notable recent exceptions (Conneau et al. 2016).”
Contextualized PACRR for Complex Answer Retrieval (2017, methodology): “MatchPyramid uses a query-document similarity matrix and a convolutional hierarchy [6].”
NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval (2018, methodology): “(Pang et al., 2016) also employ a CNN to construct the MatchPyramid model, which learns hierarchical matching patterns between local interactions of document-query pair.”
PACRR: A Position-Aware Neural IR Model for Relevance Matching (2017, methodology): “Instead, relevance matching models such as Match-Pyramid (Pang et al., 2016), DRMM (Guo et al., 2016) and the recent K-NRM (Xiong et al., 2017) resemble traditional IR retrieval measures in that they directly consider the relevance of documents’ contents with respect to the query.”
A Study of Neural Matching Models for Cross-lingual IR (2020, background): “MP, itis clear thatcosine similarityof sourcelanguage querytermand target language document term in the CLWE space is the best choice, which contradicts the conclusions in the study of monolingual IR [9] where Gaussian kernel and indicator function are found to work better. The exact matching varian”
Content-Based Weak Supervision for Ad-Hoc Re-Ranking (2017, methodology): “ocument-query term similarity histograms as input to produce a document relevance score, which is employed in Section 4. We also evaluate our weak supervision approach based on the MatchPyramid model [11], the local model from DUET (DUETL) [9], and the PACRR [7] model. All these three take querydocu”
Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search (2018, unknown): “Although other neural IR methods can improve over unsupervised baselines, feature-based learning-to-rank methods are harder to beat; end-to-end learned embeddings and matchbased techniques are necessary for current neural IR methods to provide additional improvements [22, 29, 30] Comparing the two s”
Modeling Diverse Relevance Patterns in Ad-hoc Retrieval (2018, methodology): “For deep learning methods [10, 14, 24, 26], they can be categorized into two types according to their architectures [10], namely representation-focused models and interaction-focused models.”
Training Curricula for Open Domain Answer Re-Ranking (2020, background): “Washington, DC, USA perceptron. KNRM [37] works similarly, but replaces the hard histogram buckets with soft Gaussian-kernel-based buckets. Other approaches model term adjacency, such as MatchPyramid [30], DeepRank [31], PACRR [17], and ConvKNRM [7]. Contrary to recent critiques of the actual effect”
Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval (2017, methodology): “ERR@20 [2] is employed as evaluation measure, following the configuration in the Trec Web Track [3], which is computed with the script from Trec 6 .”