Multilingual Hierarchical Attention Networks for Document Classification
By Nikolaos Pappas, Andrei Popescu-Belis. 2017
145 citations recorded by the source corpus.
Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language. However, when multilingual document collections are considered, training such models separately for each language entails linear parameter growth and lack of cross-language transfer. Learning a single multilingual model with fewer parameters is therefore a challenging but potentially beneficial objective. To this end, we propose multilingual hierarchical attention networks for learning document structures, with shared encoders and/or shared attention mechanisms across languages, using multi-task learning and an aligned semantic space as input. We evaluate the proposed models on multilingual document classification with disjoint label sets, on a large dataset which we provide, with 600k news documents in 8 languages, and 5k labels. The multilingual models outperform monolingual ones in low-resource as well as full-resource settings, and use fewer parameters, thus confirming their computational efficiency and the utility of cross-language transfer.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- James Henderson
- Dhananjay Ram
- Nikolaos Pappas
- Lesly Miculicich
- Ling Zhao
- Peng Zhou
- Tongtong Su
- Huazhi Sun
Citations in the corpus
- Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “Yin and Sch¨utze (2016) introduced multichannel embeddings and unsupervised pretraining to improve classification accuracy.”
- Privacy-preserving Neural Representations of Text (2018, background): “, 2005) and the English part of the Deutsche Welle (DW) news corpus (Pappas and Popescu-Belis, 2017).”
- A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation (2020, unknown): “[46], [47], etc.”
- A3T-GCN: Attention Temporal Graph Convolutional Network for Traffic Forecasting (2020, background): “Nowadays, attention models are widely applied in image caption generation [33], recommendation system [32], and document classification [24].”
- BAT: Deep Learning Methods on Network Intrusion Detection Using NSL-KDD Dataset (2020, methodology): “similarly to [39], attention mechanism is used to adjust probability of packet vectors so that our model pays more attention to important features.”
- Document-Level Neural Machine Translation with Hierarchical Attention Networks (2018, methodology): “For the context visualization we use the toolkit provided by Pappas and Popescu-Belis (2017).”