A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs
By Shayne Longpre, Sabeek Pradhan, Caiming Xiong, Richard Socher. 2016
10 citations recorded by the source corpus.
LSTMs have become a basic building block for many deep NLP models. In recent years, many improvements and variations have been proposed for deep sequence models in general, and LSTMs in particular. We propose and analyze a series of architectural modifications for LSTM networks resulting in improved performance for text classification datasets. We observe compounding improvements on traditional LSTMs using Monte Carlo test-time model averaging, deep vector averaging (DVA), and residual connections, along with four other suggested modifications. Our analysis provides a simple, reliable, and high quality baseline model.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Richard Socher
- Caiming Xiong
- Mohit Bansal
- Luke S. Zettlemoyer
- Graham Neubig
- Percy Liang
- Kyunghyun Cho
- Stephen Merity
Citations in the corpus
- Quasi-Recurrent Neural Networks (2016, methodology): “RNNs are also commonly the basic building block for more complex models for tasks such as machine translation (Bahdanau et al., 2015; Luong et al., 2015; Bradbury & Socher, 2016) or question answering (Kumar et al., 2016; Xiong et al., 2016).”