Long Document Classification From Local Word Glimpses via Recurrent Attention Learning

By Jun He, Liqun Wang, Liu Liu, Jiao Feng, Hao Wu. 2019

49 citations recorded by the source corpus.

Document classification requires to extract high-level features from low-level word vectors. Typically, feature extraction by deep neural networks makes use of all words in a document, which cannot scale well for a long document. In this paper, we propose to tackle the long document classification task by incorporating the recurrent attention learning framework, which can produce the discriminative features with significantly less words. Specifically, the core work is to train a recurrent neural network (RNN)-based controller, which can focus its attention on the discriminative parts. Then, the glimpsed feature is extracted by a typical short text level convolutional neural network (CNN) from the focused group of words. The controller locates its attention according to the context information, which consists of the coarse representation of the original document and the memorized glimpsed features. By glimpsing a few groups, the document can be classified by aggregating these glimpsed features and the coarse representation. For our collected 11-class 10 000-word arXiv paper data set, the proposed method outperforms two subsampled deep CNN baseline models by a large margin given much less observed words.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Muppet: Massive Multi-task Representations with Pre-Finetuning (2021, methodology): “, 2019) for classification, (2) Arxiv (He et al., 2019), PubMed (Cohan et al.”