Bidirectional recurrent neural networks

By Mike Schuster, Kuldip K. Paliwal. 1997

Subfields: Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Model Architectures

Citations in the corpus

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016, background): “Motivated by the idea of modeling differences between an intermediate layer’s output and the targets, which has shown to work well for many projects in the past [16, 21, 40], we introduce residual connections among the LSTM layers in a stack (see Figure 2).”
Neural Machine Translation by Jointly Learning to Align and Translate (2014, methodology): “With this new approach the information can be spread throughout the sequence of annotations, which can be selectively retrieved by the decoder accordingly.”
Speech recognition with deep recurrent neural networks (2013, background): “Bidirectional RNNs (BRNNs) [15] do this by processing the data in both directions with two separate hidden layers which are then fed forwards to the same output layer.”
SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders (2019, methodology): “Architecture We experimented with three base encoder-decoder conﬁgurations for SummAE: RNN-RNN, TRF-TRF, and TRF-RNN, where TRF stands for Transformer.”
Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function (2019, methodology): “The classiﬁcation model consists of an embedding layer, a bidirectional long short-term memory (BiLSTM) encoder (Hochreiter and Schmidhuber 1997; Schuster and Paliwal 1997), a max-pooling layer, a linear (fully-connected) layer, and ﬁnally a softmax layer (see Figure 1).”
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks (2015, background): “Recurrent Neural Networks (RNNs) exist in various extended forms, including bidirectional RNNs [34] that process sequences left-to-right and rightto-left in parallel.”
Improving Text-to-Image Synthesis Using Contrastive Learning (2021, methodology): “To have a fair comparison with the baselines, we adopt the same Inception-v3 [31] and Bi-directional Long Short-Term Memory (Bi-LSTM) [29] to instantiate the image encoder f and text encoder g .”
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks (2017, methodology): “Meanwhile, the last hidden states of the bi-directional LSTM are concatenated to be the global sentence vector, denoted by e ∈ RD.”
On the Evaluation of Vision-and-Language Navigation Instructions (2021, background): “The instruction representation hw is the concatenation of the final output states of a bi-directional LSTM (Schuster and Paliwal, 1997) encoding the instruction tokens W = {w1, w2, ..., wn}.”
An Actor-Critic Algorithm for Sequence Prediction (2016, methodology): “In our models, the sequence of vectors is produced by either a bidirectional RNN (Schuster & Paliwal, 1997) or a convolutional encoder (Rush et al., 2015).”
Multilingual Speech Recognition with a Single End-to-End Model (2017, methodology): “The hidden state of the decoder, d t , which captures the previous character context y t , is given by: where d t − 1 is the previous hidden state of the decoder, and ˜ y t − 1 is a character embedding vector for y t − 1 , as is typical practice in RNN-based language models.”
Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin (2015, neutral): “ striding the convolution by sframes. The goal is to shorten the number of time-steps for the recurrent layers above. Following the convolutional layers are one or more bidirectional recurrent layers [57]. The forward in time ! h land backward in time h recurrent layer activations are computed as ! ”
VaTeX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research (2019, methodology): “We employ the pretrained I3D model [13] for action recognition as the 3D ConvNet to obtain the visual features X , Bidirectional LSTM [50] (bi-LSTM) as the video encoder fenc, and LSTM [26] as the language decoder fdec.”
Deep Visual-Semantic Alignments for Generating Image Descriptions (2014, positive): “To address these concerns, we propose to use a bidirectional recurrent neural network (BRNN) [39] to compute the word representations.”
Sequence Transduction with Recurrent Neural Networks (2012, methodology): “The transcription network F is a bidirectional RNN (Schuster & Paliwal, 1997) that scans the input sequence x forwards and backwards with two separate hidden layers, both of which feed forward to a single output layer.”
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation (2018, background): “2): (1) Cascaded Encoder : The cascaded encoder aims at combining the representational power of RNNs and self-attention.”
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems (2015, background): “Previously, bidirectional networks (Schuster and Paliwal, 1997) have been shown to be effective for sequential problems (Graves et al., 2013a; Sunder-meyer et al., 2014).”
Deep Speech: Scaling up end-to-end speech recognition (2014, background): “2 The fourth layer is a bi-directional recurrent layer [38].”
Stacked Cross Attention for Image-Text Matching (2018, methodology): “Finally, we ﬁnd that using bi-directional GRU improves sentence retrieval R@1 by 4.3 and image retrieval R@1 by 0.7.”