Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

By Sheng-syun Shen, Hung-yi Lee. 2016

71 citations recorded by the source corpus.

Recurrent neural network architectures combining with attention mechanism, or neural attention model, have shown promising performance recently for the tasks including speech recognition, image caption generation, visual question answering and machine translation. In this paper, neural attention model is applied on two sequence classification tasks, dialogue act detection and key term extraction. In the sequence labeling tasks, the model input is a sequence, and the output is the label of the input sequence. The major difficulty of sequence labeling is that when the input sequence is long, it can include many noisy or irrelevant part. If the information in the whole sequence is treated equally, the noisy or irrelevant part may degrade the classification performance. The attention mechanism is helpful for sequence classification task because it is capable of highlighting important part among the entire sequence for the classification task. The experimental results show that with the attention mechanism, discernible improvements were achieved in the sequence labeling task considered here. The roles of the attention mechanism in the tasks are further analyzed and visualized in this paper.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Interpretable Reinforcement Learning Using Attention Augmented Agents (2019, background): “image classification and captioning [12, 13, 14, 15, 16, 17, 18, 19, 20], text classification [21, 22], generative models [6, 23, 24], object tracking [25], and reinforcement learning [26, 27, 28].”
Towards Robust Image Classification Using Sequential Attention Models (2019, background): “…such as question-answering [24], machine translation [6, 52], video classiﬁcation and captioning [46, 33], image classiﬁcation and captioning [37, 11, 17, 1, 55, 60, 53, 5, 57], text classiﬁcation [58, 47], generative models [42, 59, 30], object tracking [29], and reinforcement learning [10].”