Semi-supervised Question Retrieval with Gated Convolutions

By Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi S. Jaakkola, Kateryna Tymoshenko, Alessandro Moschitti, Lluís Màrquez i Villodre. 2015

112 citations recorded by the source corpus.

Question answering forums are rapidly growing in size with no effective automated ability to refer to and reuse answers already available for previous posted questions. In this paper, we develop a methodology for finding semantically related questions. The task is difficult since 1) key pieces of information are often buried in extraneous details in the question body and 2) available annotations on similar questions are scarce and fragmented. We design a recurrent and convolutional model (gated convolution) to effectively map questions to their semantic representations. The models are pre-trained within an encoder-decoder framework (from body to title) on the basis of the entire raw corpus, and fine-tuned discriminatively from limited annotations. Our evaluation demonstrates that our model yields substantial gains over a standard IR baseline and various neural network architectures (including CNNs, LSTMs and GRUs).

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

A Retrieve-and-Edit Framework for Predicting Structured Outputs (2018, background): “Some question answering systems learn to retrieve based on supervision of the correct item to retrieve [35, 27, 19], but these approaches do not apply to our setting since we do not know which items are easy to edit into our target sequence y and must instead estimate this from the embedding.”
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (2016, background): “, document ranking [27], sentiment analysis [18], and question retrieval [22]).”
Deriving Neural Architectures from Sequence and Graph Kernels (2017, background): “This includes recurrent modules with the ability to carry persistent memories such as LSTM (Hochreiter & Schmid-huber, 1997) and GRU (Chung et al., 2014), as well as non-consecutive convolutional modules (RCNNs, Lei et al. (2015)), and others.”
Unsupervised Question Answering by Cloze Translation (2019, background): “Wang et al. (2018) consider semi-supervised cloze QA, Chen et al. (2018) use semi-supervision to improve semantic parsing on WebQuestions (Berant et al., 2013), and Lei et al. (2016) leverage semi-supervision for question similarity modelling.”
Multi-Stage Pretraining for Low-Resource Domain Adaptation (2020, methodology): “For AskUbuntu-DQD, we report MAP, MRR, Precision@1 and Precision@5 following (Lei et al., 2016).”
Training RNNs as Fast as CNNs (2017, background): “SRU can be viewed as a simpliﬁed version (or special case) of Quasi-RNN, RCNN and KNN with window size 1 and additional highway connections.”
Neural information retrieval: at the end of the early years (2017, background): “…et al. (2014) 9.3.1 Similar Question Retrieval Aggregate Explicit Zhou et al. (2015) 9.3.2 Learn Learn to match Lei et al. (2016) Learn to generate Lei et al. (2016) 9.4.1 Content-based recommendation Aggregate Explicit Manotumruksa et al. (2016) 9.4.2 Learn Learn to match Gao et al. (2014) 10…”
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale (2020, background): “Zero-shot transfer of text matching models is particularly challenging in setups of non-factoid answer selection (Cohen et al., 2018; Tay et al., 2017; Feng et al., 2015; Verberne et al., 2010) and question similarity (Nakov et al., 2017; Lei et al., 2016).”
Multilingual Universal Sentence Encoder for Semantic Retrieval (2019, methodology): “Table 2 shows the MAP@100 on the Quo-ra/AskUbuntu retrieval tasks.”
Effective Transfer Learning for Identifying Similar Questions: Matching User Questions to COVID-19 FAQs (2020, result): “Medical Question-Question similarity While there has been significant work on question-question similarity ([7, 18] and references therein), research in medical question-question similarity is still somewhat nascent. closest to our work is that of Abacha and Demner-Fushman [3, 4].”
Neural Paraphrase Identification of Questions with Noisy Pretraining (2017, background): “Lei et al. (2016) consider a related task leveraging the AskUbuntu corpus (dos Santos et al., 2015), but it contains two orders of magnitude less annotations, thus limiting the quality of any model.”
A question-entailment approach to question answering (2019, background): “[30] proposed a recurrent and convolutional model (gated convolution) to map questions to their semantic representations.”