Neural Paraphrase Identification of Questions with Noisy Pretraining

By Gaurav Singh Tomar, Thyago Duque, Oscar Täckström, Jakob Uszkoreit, Dipanjan Das. 2017

81 citations recorded by the source corpus.

We present a solution to the problem of paraphrase identification of questions. We focus on a recent dataset of question pairs annotated with binary paraphrase labels and show that a variant of the decomposable attention model (replacing the word embeddings of the decomposable attention model of Parikh et al. 2016 with character n-gram representations) results in accurate performance on this task, while being far simpler than many competing neural architectures. Furthermore, when the model is pretrained on a noisy dataset of automatically collected question paraphrases, it obtains the best reported performance on the dataset.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

BLEURT: Learning Robust Metrics for Text Generation (2020, background): “To our knowledge no existing work has explored pre-training and extrapolation in the context of NLG. Noisy pre-training has been proposed before for other tasks such as paraphrasing (Wieting et al., 2016; Tomar et al., 2017) but generally not with synthetic data.”
Natural Language Inference over Interaction Space (2017, neutral): “(ii) Joint feature models which use the cross sentence feature or attention from one sentence to another(Rockt¨aschel et al., 2015; Wang & Jiang, 2015; Cheng et al., 2016; Parikh et al., 2016; Wang et al., 2017; Yu & Munkhdalai, 2017; Sha et al., 2016).”
Identifying Well-formed Natural Language Questions (2018, methodology): “…are available tools and datasets, such as Quora question paraphrases and the Par-alex dataset (Fader et al., 2013) – for identifying query paraphrases (Wang et al., 2017; Tomar et al., 2017), but these datasets do not contain information about whether a query is a natural language question or not.”
The BQ Corpus: A Large-scale Domain-specific Chinese Corpus For Sentence Semantic Equivalence Identification (2018, background): “Tomar et al. (2017) proposed a variant of the decomposable attention model.”
Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information (2018, unknown): “…LSTM (Wang, Hamza, and Florian 2017) 83.21 L.D.C. (Wang, Hamza, and Florian 2017) 85.55 BiMPM (Wang, Hamza, and Florian 2017) 88.17 pt-DecAttchar.c (Tomar et al. 2017) 88.40 DIIN (Gong, Luo, and Zhang 2018) 89.06 DRCN 90.15 DIIN* (Gong, Luo, and Zhang 2018) 89.84 DRCN * 91.30 (He, Gimpel, and Lin…”
LCQMC:A Large-scale Chinese Question Matching Corpus (2018, methodology): “Second, we concatenate the representations of both sentences and use softmax to make a classiﬁcation (Mueller and Thyagarajan, 2016; Tomar et al., 2017).”
Stochastic Answer Networks for Natural Language Inference (2018, unknown): “(Tomar et al., 2017) 88.4 (Gong et al., 2017) 89.1 SAN 89.4 SciTail Dataset (Accuracy”