Sentence Similarity Learning by Lexical Decomposition and Composition

By Zhiguo Wang, Haitao Mi, Abraham Ittycheriah. 2016

207 citations recorded by the source corpus.

Most conventional sentence similarity methods only focus on similar parts of two input sentences, and simply ignore the dissimilar parts, which usually give us some clues and semantic meanings about the sentences. In this work, we propose a model to take into account both the similarities and dissimilarities by decomposing and composing lexical semantics over sentences. The model represents each word as a vector, and calculates a semantic matching vector for each word based on all words in the other sentence. Then, each word vector is decomposed into a similar component and a dissimilar component based on the semantic matching vector. After this, a two-channel CNN model is employed to capture features by composing the similar and dissimilar components. Finally, a similarity score is estimated over the composed feature vectors. Experimental results show that our model gets the state-of-the-art performance on the answer sentence selection task, and achieves a comparable result on the paraphrase identification task.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Key-Value Memory Networks for Directly Reading Documents (2016, methodology): “Key-Value Memory Networks outperform a large set of other meth-ods, although the L.D.C. method of (Wang et al. , 2016) is only slightly worse.”
Multi-Perspective Context Matching for Machine Comprehension (2016, methodology): “examples in Table 1). Therefore, we deﬁne the ﬁlter layer to ﬁlter out redundant information from the passage. First, we calculate a relevancy degree r j for each word p j in passage P. Inspired from Wang et al. (2016b), we compute the relevancy degree r i;j between each word pair q i 2Q and p j 2P ”
Exploiting Source-side Monolingual Data in Neural Machine Translation (2016, background): “, 2014) or sentence embedding (Zhang et al., 2015; Wang et al., 2016a; Wang et al., 2016b) to select the relevant monolingual data.”
Bilateral Multi-Perspective Matching for Natural Language Sentences (2017, methodology): “We experiment on two datasets: TREC-QA [ Wang et al. , 2007 ] and WikiQA [ Yang et al. , 2015 ] .”
Neural Cross-Lingual Entity Linking (2017, methodology): “…then use a combination of convolutional neural networks (CNN), LSTMs (Hochreiter and Schmidhuber 1997), Lexical Composition and Decomposition (Wang, Mi, and Ittycheriah 2016), Multi-Perspective Context Matching (MPCM) (Wang et al. 2016), and Neural Tensor Networks (Socher et al. 2013a;…”
Neural Paraphrase Identification of Questions with Noisy Pretraining (2017, background): “Paraphrase identification is a well-studied task in NLP (Das and Smith, 2009; Chang et al., 2010; He et al., 2015; Wang et al., 2016, inter alia).”
The BQ Corpus: A Large-scale Domain-specific Chinese Corpus For Sentence Semantic Equivalence Identification (2018, background): “…convolutional neural networks (DCNNs), recurrent neural networks (RNNs), and their variants, such as Arc-I, Arc-II and BiMPM etc., have been developed and verified on it, even though it contains only thousands of sentence pairs (Hu et al., 2014; Yin and Schütze, 2015; Wang et al., 2016, 2017).”
Adaptive Convolutional Filter Generation for Natural Language Understanding (2017, methodology): “Baselines We also compare our ACNN models with twelve strong baseline methods, categorized as follows: (i) Word Cnt, Wgt Word Cnt, LCLR (Yang, Yih, and Meek, 2015): these methods leverage lexical semantic features for prediction; (ii) Paragraph vector (PV) (Le and Mikolov, 2014): an unsupervised alg”
Analysis of Wikipedia-based Corpora for Question Answering (2018, background): “…(Shen and Klakow, 2006; Sultan et al., 2016) finds answer phrases whereas answer selection (Wang et al., 2007; Yih et al., 2013; Yu et al., 2014; Wang et al., 2016) and answer triggering (Yang et al., 2015; Jurczyk et al., 2016) find answer sentences instead, although the presence of the answer…”
Evolution of Semantic Similarity—A Survey (2020, background): “• Wang et al.[97] proposed a model to estimate semantic similarity between two sentences based on lexical decomposition and composition.”
Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering (2018, unknown): “Negative information has also been show to be useful in answer sentence selection [28, 30].”
Learning Context-Sensitive Convolutional Filters for Text Processing (2017, background): “Wang et al. (2016); Wang and Jiang (2017) explore a compare and aggregate framework to directly capture the word-by-word matching between two paired sentences.”