Adaptive Convolutional Filter Generation for Natural Language Understanding

By Dinghan Shen, Martin Renqiang Min, Yitong Li, Lawrence Carin. 2017

11 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP are not expressive enough, in the sense that all input sentences share the same learned (and static) set of filters. Motivated by this problem, we propose an adaptive convolutional filter generation framework for natural language understanding, by leveraging a meta network to generate input-aware filters. We further generalize our framework to model question-answer sentence pairs and propose an adaptive question answering (AdaQA) model; a novel two-way feature abstraction mechanism is introduced to encapsulate co-dependent sentence representations. We investigate the effectiveness of our framework on document categorization and answer sentence-selection tasks, achieving state-of-the-art performance on several benchmark datasets.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Deconvolutional Latent-Variable Model for Text Sequence Matching (2017, background): “With recent advances in deep neural networks, considerable research has focused on developing end-to-end deep learning models for text sequence matching (Hu et al. 2014; Wang and Jiang 2017; Rocktäschel et al. 2015; Wang, Hamza, and Florian 2017; Shen et al. 2017).”
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing (2018, methodology): “Inspired by the recent success of variational autoencoders for various NLP problems (Miao et al., 2016; Bowman et al., 2015; Yang et al., 2017; Miao et al., 2017; Shen et al., 2017b; Wang et al., 2018), we approach the training of discrete (binary) latent variables from a generative perspec- tive.”
Video Generation From Text (2017, background): “2016) for video prediction and image-style transformation, and (Shen et al. 2017) for question answering.”
Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information (2018, background): “Most competitive models (Shen, Yang, and Deng 2017; Bian et al. 2017; Wang, Hamza, and Florian 2017; Shen et al. 2017) also use attention methods for words alignment between question and candidate answer sentences.”