Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks

By Hua He, Kevin Gimpel, Jimmy J. Lin. 2015

383 citations recorded by the source corpus.

Modeling sentence similarity is complicated by the ambiguity and variability of linguistic expression. To cope with these challenges, we propose a model for comparing sentences that uses a multiplicity of perspectives. We first model each sentence using a convolutional neural network that extracts features at multiple levels of granularity and uses multiple types of pooling. We then compare our sentence representations at several granularities using multiple similarity metrics. We apply our model to three tasks, including the Microsoft Research paraphrase identification task and two SemEval semantic textual similarity tasks. We obtain strong performance on all tasks, rivaling or exceeding the state of the art without using external resources such as WordNet or parsers.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation (2017, background): “…semantic representations (i.a., Arora et al. (2017); Conneau et al. (2017); Mu et al. (2017); Pagliardini et al. (2017); Wieting and Gimpel (2017); He and Lin (2016); Hill et al. (2016); Kenter et al. (2016); Lau and Baldwin (2016); Wieting et al. (2016a,b); He et al. (2015); Pham et al. (2015)).”
Efficient Natural Language Response Suggestion for Smart Reply (2017, methodology): “Networks built on n-gram embeddings such as those presented in this paper (see section 4) are computationally inexpensive relative to RNN and convolutional network [6, 30] encoders.”
Ethical Challenges in Data-Driven Dialogue Systems (2017, methodology): “The semantic similarities are measured by word embedding cosine distance, LSTM similarity model (Mueller and Thya-garajan 2016) and CNN similarity models (He, Gimpel, and Lin 2015).”
Natural Language Inference by Tree-Based Convolution and Heuristic Matching (2015, methodology): “Although element-wise distance is used to detect paraphrase in He et al. (2015), it mainly reﬂects “similarity” information.”
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, neutral): “…representative neural network (NN) approaches and non-NN approaches: (i) A-LSTM; (ii) A-LSTM(+): A-LSTM plus hand-crafted features; (iii) RAE (Socher et al., 2011), recursive autoencoder; (iv) Bi-CNN-MI (Yin and Sch¨utze, 2015a MPSSM-CNN (He et al., 2015), the state-of-the-art NN system for PI.”
Charagram: Embedding Words and Sentences via Character n-grams (2016, methodology): “Most work uses words as the smallest units in the compositional architecture, often using pretrained word embeddings or learning them speciﬁcally for the task of interest (Tai et al., 2015; He et al., 2015).”
SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation (2016, methodology): “The team extends a multi-perspective convolutional neural network (MPCNN) (He et al., 2015) with a simple word level attentional mecha- 17To see how much of this is related to the Q&A domain in particular, we will investigate including difficult non-Q&A evaluation data in future STS…”
Siamese Recurrent Architectures for Learning Sentence Similarity (2016, neutral): “…0.8655 0.7995 0.2561 (Kiros et al. 2015) Dependency Tree-LSTM 0.8676 0.8083 0.2532 (Tai, Socher, and Manning 2015) ConvNet 0.8686 0.8047 0.2606 (He, Gimpel, and Lin 2015) MaLSTM 0.8822 0.8345 0.2286 In Table 3, Tai, Socher, and Manning show the most similar test-set examples found by their…”
Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement (2016, methodology): “Neural networks and distributed representations can alleviate such sparsity, thus neural network-based models are widely used by recent systems for the STS problem (He et al., 2015; Tai et al., 2015; Yin and Sch¨utze, 2015).”
Rethinking Search: Making Experts out of Dilettantes (2021, neutral): “Due to the nature of how the task is defined, systems typically rely on advances in short text matching [73], paraphrase identification [33], and entailment detection [63, 84].”
Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer (2018, background): “Recently, He et al., (2015) (H E 15 in Table 4) developed a convolutional neural network based sentence similarity measure.”
Automatically Extracting High-Quality Negative Examples for Answer Selection in Question Answering (2017, neutral): “[6], which also has open-source code available.”
Learning to Paraphrase for Question Answering (2017, positive): “We estimate p θ ( q (cid:48) | q ) using neural networks given their successful application to paraphrase identiﬁcation tasks (Socher et al., 2011; Yin and Sch¨utze, 2015; He et al., 2015).”
Neural Paraphrase Identification of Questions with Noisy Pretraining (2017, background): “Paraphrase identification is a well-studied task in NLP (Das and Smith, 2009; Chang et al., 2010; He et al., 2015; Wang et al., 2016, inter alia).”
Deep Learning--based Text Classification (2020, methodology): “rity. Liu et al. [120] model the interaction of a sentence pair with two coupled-LSTMs. In addition to RNNs, BOW models and CNNs are also used in S2Nets to represent sentences. For example, He et al. [121] propose a S2Net that uses CNNs to model multi-perspective sentence similarity. Renter et al. [”
Semantic Text Matching for Long-Form Documents (2019, background): “For instance, some studies [2, 27, 32, 33, 45, 46] exploit RNNs to model the sequential information of documents while other work [19, 38, 46, 54] applies CNNs to capture the important local messages in documents for semantic text matching.”
CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain (2019, background): “The Siamese structure (Bromley et al., 1994) for metric learning achieve great success and is widely applied (Amiri et al., 2016; Liu et al., 2018; Mueller and Thyagarajan, 2016; Neculoiu et al., 2016; Wan et al., 2016; He et al., 2015).”
HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity (2017, methodology): “5) All sentences are padded to a static length l = 30 with zero vectors (He et al., 2015a).”