A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

By Shengxian Wan, Yanyan Lan, Jiafeng Guo, Jun Xu, Liang Pang, Xueqi Cheng. 2015

359 citations recorded by the source corpus.

Matching natural language sentences is central for many applications such as information retrieval and question answering. Existing deep models rely on a single sentence representation or multiple granularity representations for matching. However, such methods cannot well capture the contextualized local information in the matching process. To tackle this problem, we present a new deep architecture to match two sentences with multiple positional sentence representations. Specifically, each positional sentence representation is a sentence representation at this position, generated by a bidirectional long short term memory (Bi-LSTM). The matching score is finally produced by aggregating interactions between these different positional sentence representations, through k-Max pooling and a multi-layer perceptron. Our model has several advantages: (1) By using Bi-LSTM, rich context of the whole sentence is leveraged to capture the contextualized local information in each positional sentence representation; (2) By matching with multiple positional sentence representations, it is flexible to aggregate different important contextualized local information in a sentence to support the matching; (3) Experiments on different tasks such as question answering and sentence completion demonstrate the superiority of our model.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

A Deep Relevance Matching Model for Ad-hoc Retrieval (2016, methodology): “One is dynamic pooling as in [25, 19] which keeps the position information, and the other is Kmax pooling as in [26] which turns the positional signals into strength related signals.”
Natural Language Inference by Tree-Based Convolution and Heuristic Matching (2015, methodology): “Wan et al. (2015) build sentence pair models upon recurrent neural networks (RNNs) to iteratively integrate information along a sentence.”
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks (2019, neutral): “These generic architectures can be applied to tasks like sentence classiﬁcation (Zhang et al., 2015; Conneau et al., 2016) and sentence matching (Wan et al., 2016; He et al., 2016), but the model is trained only on data of a particular task.”
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, background): “Wan et al. (2015) try to match two sentences in AS and SC by multi-”
Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models (2019, methodology): “[9], CDSSM [18], DRMM [7], KNRM [21], DUET [15]. These models were selected because they were specifically designed forad hoc retrieval; other models available in MatchZoo, such as ARC-I [8], MV-LSTM [20], andaNMM[22]weremainlydesignedforshorttextsandnotgeared towards handling documents (which are m”
Word Embedding based Edit Distance (2018, methodology): “Specially, many supervised methods have been developed recently using deep neural networks Shen et al. (2014), Hu et al. (2014), Pang et al. (2016), Wan et al. (2016), Parikh et al. (2016).”
A Deep Look into Neural Ranking Models for Information Retrieval (2019, methodology): “• Recurrent networks have been used for φ and ψ in LSTM-RNN [48] and MV-LSTM [88].”
Induction Networks for Few-Shot Text Classification (2019, methodology): “, 2013) in this module, which has shown great advantages in modeling the relationship between two vectors (Wan et al., 2016; Geng et al., 2017).”
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications (2019, methodology): “…bilinear similarity features, CNTN (Qiu and Huang, 2015) using neural tensor network, LSTM (Tay et al., 2017) us-ing single and multi-layer, MV-LSTM (Wan et al., 2016), NTN-LSTM and HD-LSTM (Tay et al., 2017) using holographic dual LSTM and Capsule-Zhao (Zhao et al., 2018) using capsule networks.”
Sequential Attention-based Network for Noetic End-to-End Response Selection (2019, methodology): “…and Kleindienst 2015) 0.638 0.784 0.949 0.365 0.536 0.828 BiLSTM (Kadlec, Schmid, and Kleindienst 2015) 0.630 0.780 0.944 0.355 0.525 0.825 MV-LSTM (Wan et al. 2016) 0.653 0.804 0.946 0.412 0.591 0.857 Match-LSTM (Wang and Jiang 2016) 0.653 0.799 0.944 0.410 0.590 0.858 Attentive-LSTM (Tan, Xiang,…”
Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN (2016, methodology): “Examples include RAE [Socher et al., 2011], DSSM [Huang et al., 2013], CDSSM [Shen et al., 2014], ARC-I [Hu et al., 2014], CNTN [Qiu and Huang, 2015], LSTM-RNN [Palangi et al., 2015], MultiGranCNN [Yin and Schütze, 2015a; Yin and Schütze, 2015b] and MV-LSTM [Wan et al., 2016].”
CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain (2019, background): “The Siamese structure (Bromley et al., 1994) for metric learning achieve great success and is widely applied (Amiri et al., 2016; Liu et al., 2018; Mueller and Thyagarajan, 2016; Neculoiu et al., 2016; Wan et al., 2016; He et al., 2015).”
RiFeGAN: Rich Feature Generation for Text-to-Image Synthesis From Prior Knowledge (2020, background): “[29] propose a deep architecture exploiting positional sentence representations, generated by a bidirectional long short term memory (Bi-LSTM), k-Max pooling, and a multi-layer perceptron, to match two sentences.”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, methodology): “An alternative strategy, in contrast, is to combine the representation vectors of query and response first, and then a neural method is used on the combined representation pair to further learn the interaction [180, 173, 126, 105].”
Convolutional Interaction Network for Natural Language Inference (2018, background): “Generally, NLI is also related to many other NLP tasks under the paradigm of semantic matching of two texts, such as question answering Hu et al. (2014); Wan et al. (2016) and information retrieval Liu et al. (2015), and so on.”
When large language models meet personalization: perspectives of challenges and opportunities (2023, neutral): “Generative methods [198–200] utilize a sequence-to-sequence model structure to generate responses, while retrieval-based methods [201–203] transform the task of generating responses into a retrieval problem by searching for the most relevant response in a response database based on…”
Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction (2018, background): “They also observed that such a structure implements an implicit form of attention, by producing an input encoding that depends on the output sequence produced so far.”