Speed Reading: Learning to Read ForBackward via Shuttle

By Tsu-Jui Fu, Wei-Yun Ma. 2018

17 citations recorded by the source corpus.

We present LSTM-Shuttle, which applies human speed reading techniques to natural language processing tasks for accurate and efficient comprehension. In contrast to previous work, LSTM-Shuttle not only reads shuttling forward but also goes back. Shuttling forward enables high efficiency, and going backward gives the model a chance to recover lost information, ensuring better prediction. We evaluate LSTM-Shuttle on sentiment analysis, news classification, and cloze on IMDB, Rotten Tomatoes, AG, and Children’s Book Test datasets. We show that LSTM-Shuttle predicts both better and more quickly. To demonstrate how LSTM-Shuttle actually behaves, we also analyze the shuttling operation and present a case study.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiji Song
2. Yulin Wang
3. Gao Huang
4. Julian McAuley
5. Le Yang
6. Yizeng Han
7. Canwen Xu
8. Honghui Wang

In Natural Language Processing

1. Gao Huang
2. Shiji Song
3. Jesse G. Beu
4. Jin Tao
5. Ganesh S. Dasika
6. Julian McAuley
7. Yulin Wang
8. Canwen Xu

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, background): “Language Sa & Te Language modeling [11], [16], [118], [170], [172], Machine translation [16], [35], [36], Classification [64], [65], [174], Processing Sentiment analysis [168], [169], [171], [175], [176], Question answering [35], [63], [168], [171], [173]”
• Skipping RNN State Updates without Retraining the Original Model (2019, methodology): “There have been multiple works on skipping RNN state updates [1, 4, 11, 17, 18].”
• Neural Speed Reading with Structural-Jump-LSTM (2019, unknown): “This is often denoted as speed reading : obtaining the same accuracy while using (far) fewer FLOPs (Yu et al., 2017; 2018; Seo et al., 2018; Huang et al., 2017; Fu & Ma, 2018).”
• A Survey on Dynamic Neural Networks for Natural Language Processing (2022, background): “…sentence; hidden states stop reading RIM (Li et al., 2019) input sentence; hidden states skip a single step; stop reading Yu et al. (2018) hidden states skip multiple steps; stop reading; re-read LSTM-Shuttle (Fu and Ma, 2018) hidden states skip multiple steps; jump back multiples steps Struc.”