Convolutional Neural Network Architectures for Matching Natural Language Sentences

By Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen. 2014

1,380 citations recorded by the source corpus.

Semantic matching is of central importance to many natural language tasks [2,28]. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

A Decomposable Attention Model for Natural Language Inference (2016, background): “A large body of work based on neural networks for text similarity tasks including NLI have been published in recent years (Hu et al., 2014; Rockt¨aschel et al., 2016; Wang and Jiang, 2015; Yin et al., 2016, inter alia ).”
End-to-End Neural Ad-hoc Ranking with Kernel Pooling (2017, background): “ocumentsusing their embedding similarity with the query. „e interaction based neural models, on the other hand, learn query-document matching pa−erns from word-level interactions. For example, ARC-II [10] and MatchPyramid [20] build hierarchical Convolutional Neural Networks (CNN) on the interaction”
A Deep Relevance Matching Model for Ad-hoc Retrieval (2016, methodology): “In ARC-II [11] and MatchPyramid [19], Φ maps each text to a sequence of word vectors,”
Learning to Extract Coherent Summary via Deep Reinforcement Learning (2018, methodology): “This model is built on the ARC-II proposed by (Hu et al. 2014) for sentence matching.”
Hierarchical Question-Image Co-Attention for Visual Question Answering (2016, methodology): “Our pooling method differs from those used in previous works [9] in that it adaptively selects different gram features at each time step, while preserving the original sequence length and order.”
Towards Universal Paraphrastic Sentence Embeddings (2015, methodology): “…on non-syntactic hierarchical structure (Zhao et al., 2015; Chen et al., 2015b), convolutional neural networks (Kalchbrenner et al., 2014; Kim, 2014; Hu et al., 2014; Yin & Sch¨utze, 2015; He et al., 2015), and recurrent neural networks using long short-term memory (Tai et al., 2015; Ling et al.,…”
Reasoning about Entailment with Neural Attention (2015, background): “Despite the success of neural networks for paraphrase detection (e.g. Socher et al., 2011; Hu et al., 2014; Yin and Schütze, 2015), end-to-end differentiable neural architectures failed to get close to acceptable performance for RTE due to the lack of large high-quality datasets.”
Self-Adaptive Hierarchical Sentence Model (2015, background): “…representations and deep neural networks, it is now common practice to find a vectorial representation of sentences, which turns out to be quite effective for tasks of classification [Kim, 2014], machine translation [Cho et al., 2014; Bahdanau et al., 2015], and semantic matching [Hu et al., 2014].”
Natural Language Inference over Interaction Space (2017, methodology): “, 2017), sentence classification(Kim, 2014), text matching(Hu et al., 2014) and sentiment analysis(Kalchbrenner et al.”
Natural Language Inference by Tree-Based Convolution and Heuristic Matching (2015, methodology): “A dynamic pooling layer is applied to aggregate information along different parts of the tree, serving as a way of semantic compositionality (Hu et al., 2014).”
Learning to Answer Questions from Image Using Convolutional Neural Network (2015, methodology): “CNNs (Hu et al. 2014; Kim 2014; Kalchbrenner, Grefenstette, and Blunsom 2014) also demonstrate their powerful abilities on the sentence representation for paraphrase, sentiment analysis, and so on.”
Recent Trends in Deep Learning Based Natural Language Processing (2017, background): “CNN models are also suitable for certain NLP tasks that require semantic matching beyond classification (Hu et al., 2014).”
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms (2018, methodology): “For MSRP dataset, we follow the setup in Hu et al. (2014) and do not use any additional features.”
Representation learning for very short texts using weighted word embedding aggregation (2016, methodology): “One possibility is to use an multilayer perceptron (MLP) with the whole sentence as an input, or a 1D convolutional neural network (Collobert et al. (2011); Hu et al. (2014); Xu et al. (2015); Johnson and Zhang (2015)).”
Multimodal Convolutional Neural Networks for Matching Image and Sentence (2015, methodology): “Specifically, convolutional neural networks (CNNs) have shown their powerful abilities on learning of image representation [10, 29, 33, 11] and sentence representation [14, 15, 18].”
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, background): “Hu et al. (2014) present two CNN architectures, ARC-I and ARC-II, for sentence matching.”
How Transferable are Neural Networks in NLP Applications? (2016, methodology): “Such a two-step strategy is called the “Siamese” structure (Bromley et al., 1993), and also used for various sentence pair modeling applications (Hu et al., 2014; Mou et al., 2015b).”
Adversarial Feature Matching for Text Generation (2017, methodology): “There are other CNN architectures in the literature (Kalchbrenner et al., 2014; Hu et al., 2014; Johnson & Zhang, 2015).”
Convolutional Neural Network for Paraphrase Identification (2015, methodology): “, ARC-I, Hu et al. (2014)).”
LCSTS: A Large Scale Chinese Short Text Summarization Dataset (2015, background): “Recently, deep learning methods have shown potential abilities to learn representation (Hu et al., 2014; Zhou et al., 2015) and generate language (Bahdanau et al.”