Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search

By Zhuyun Dai, Chenyan Xiong, Jamie Callan, Zhiyuan Liu. 2018

352 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Information Retrieval & Search, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

In Information Retrieval & Search

In Model Architectures

Citations in the corpus

XLNet: Generalized Autoregressive Pretraining for Language Understanding (2019, methodology): “Following the setting in previous work [8], we use the ClueWeb09-B dataset to evaluate the performance on document ranking.”
Passage Re-ranking with BERT (2019, background): “AR MRR@10 MAP Method Dev Eval Test BM25 (Lucene, no tuning) 16.7 16.5 12.3 BM25 (Anserini, tuned) - - 15.3 Co-PACRR? (MacAvaney et al., 2017) - - 14.8 KNRM (Xiong et al., 2017) 21.8 19.8 - Conv-KNRM (Dai et al., 2018) 29.0 27.1 - IRNety 27.8 28.1 - BERT Base 34.7 - 31.0 BERT Large 36.5 35.8 33.5 Tab”
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (2020, background): “To begin with, we notice the fast progress from KNRM in 2017 to the BERT-based models in 2019, manifesting itself in over 16% increase in MRR@10.”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “oup (2) provides results from a few pre-BERT interaction-based neural ranking models. Rows (2a) and (2b) are taken directly from Hofstätter et al. [2020] for Co-PACRR [Hui et al., 2018] and ConvKNRM [Dai et al., 2018]. Row (2c) is taken from Hofstätter et al. [2019], which provides more details on t”
Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval (2019, methodology): “Duet V2 [20] 0.243 +0% 0.245 +2% Conv-KNRM [9] 0.247 +2% 0.247 +3% FastText+Conv-KNRM [14] 0.277 +14% 0.290 +21% BERT Re-Ranker [22] 0.365 +50% 0.359 +50% depths, meaning a ranking from DeepCT-Index provided more relevant passages to a re-ranker.”
Understanding the Behaviors of BERT in Ranking (2019, methodology): “In comparison, Conv-KNRM (Bing), the same ConvKNRM model but pre-trained on Bing user clicks [1], performs the best on ClueWeb, and much better than Conv-KNRM pretrained on MARCO labels.”
Context-Aware Document Term Weighting for Ad-Hoc Search (2020, methodology): “The first, LeToR , is a popular feature-based learning-to-rank method using Coordinate Ascent [9, 11, 31].”
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations (2020, methodology): “[21] demonstrate that Locality-Sensitive Hashing (LSH) and other tricks can be employed to improve the performance of interaction-focused methods such as DRMM [10], KNRM [44], and ConvKNRM [5].”
Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking (2020, methodology): “oding and sequence wide self-attention allows for local and global contextualization at the same time. This makes TK more powerful than previous local-only contextualization methods used in CONV-KNRM [5] and CO-PACRR [16]. ... Query Document Contextualized encoding Term x Term interaction Kernels”
EARL: Speedup Transformer-based Rankers with Pre-computed Representation (2020, methodology): “Recent innovations in neural rankering models have advanced the state-of-the-art retrieval performance [6, 7, 14, 26].”
Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering (2020, background): “xed similarity/distance measure (cosine or Euclidean) but researchers have explored others strategies such as siamese networks (Das et al. 2016), histograms (Guo et al. 2016), convolutional networks (Dai et al. 2018), etc. In the recent litterature (MacAvaney et al. 2019), several combinations betwe”
Capreolus: A Toolkit for End-to-End Neural Ad Hoc Retrieval (2020, methodology): “interaction-focused models (DRMM [4], KNRM [16], PACRR [7], ConvKNRM [2], POSITDRMM [12]), hybrid models (DUET [11]), and hierarchical models (HiNT [3] and DeepTileBars [15]).”
Efficient Interaction-based Neural Ranking with Locality Sensitive Hashing (2019, methodology): “Neural networks have been applied in information retrieval and text mining tasks (e.g. [5, 9, 20, 23, 24, 33–36]).”
Document Re-Ranking Model for Machine-Reading and Comprehension (2020, background): “[9] pointed out that DRMM returns inconsistent similarities based on the lengths of the query and document vectors.”
A Modern Perspective on Query Likelihood with Deep Generative Retrieval Models (2021, neutral): “Arguably, all existing neural ranking models [7, 11, 12, 14, 25, 28, 33, 50] follow the text matching paradigm, where relevance is calculated as the predicted matching score of each document to a given query.”
Promptagator: Few-shot Dense Retrieval From 8 Examples (2022, methodology): “Interaction based models on the other hand explicitly model the interaction between query and document terms (Guo et al., 2016; Hui et al., 2017; Xiong et al., 2017; Dai et al., 2018; McDonald et al., 2018; Nogueira & Cho, 2019), and therefore make more informed decisions.”
Kernel Graph Attention Network for Fact Verification (2019, methodology): “One of the effective ways to model text matches is to leverage matching kernels (Xiong et al., 2017; Dai et al., 2018), which summarize word or phrase interactions in the learned embedding space between query and documents.”
Selective Weak Supervision for Neural Information Retrieval (2020, methodology): “In scenarios with sufficient training signals, for example, in commercial search engines with large amounts of user clicks, and on benchmarks with millions of relevance labels, end-to-end Neu-IR methods have significantly improved their ranking accuracy [5, 23, 29, 45].”
Cross-Domain Sentence Modeling for Relevance Transfer with BERT (2019, methodology): “For “ad hoc” document ranking, they explore using BERT both as representation-based and interaction-based rankers and in combination with KNRM [65] and Conv-KNRM [16].”