Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation

By Tianyu He, Xu Tan, Yingce Xia, Di He, Tao Qin, Zhibo Chen, Tie-Yan Liu. 2018

125 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, neutral): “…et al., 2015a; Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017; Tan et al., 2019; Artetxe et al., 2017; Lample et al., 2017; 2018; He et al., 2018; Hassan et al., 2018; Song et al., 2018; Shen et al., 2018), text summarization (Ayana et al., 2016; Suzuki & Nagata, 2017; Gehring…”
Learning Deep Transformer Models for Machine Translation (2019, neutral): “In contrast, our approach allows the encoder layers to interact earlier, which has been proven to be effective in machine translation (He et al., 2018) and text match (Lu and Li, 2013).”
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning (2020, neutral): “35 LYC Transforemer (He et al., 2018) 35.”
Multilingual Neural Machine Translation with Knowledge Distillation (2019, background): “Neural Machine Translation (NMT) has witnessed rapid development in recent years (Bahdanau et al., 2015; Luong et al., 2015b; Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017; Wu et al., 2018; Song et al., 2018; Shen et al., 2018; Guo et al., 2018; He et al., 2018; Gong et al., 2018), inc”
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks (2019, methodology): “Additionally, we experimented with a layer-wise attention mechanism (He et al., 2018), but got nearly identical numbers on most tasks.”
Multilingual Neural Machine Translation with Language Clustering (2019, background): “…(Bahdanau et al., 2015; Luong et al., 2015b; Sutskever et al., 2014; Wu et al., 2016; Gehring et al., 2017; Vaswani et al., 2017; Shen et al., 2018; He et al., 2018) is adopted to model the conditional probability P ( y | x, θ ) , where the encoder maps the input to a set of hidden representations…”
Towards Mitigating Gender Bias in a decoder-based Neural Machine Translation model by Adding Contextual Information (2020, methodology): “The architecture in our experiments uses only the decoder part of the popular Transformer (Vaswani et al., 2017; He et al., 2018; Fonollosa et al., 2019); thus, reduces training parameters and simpliﬁes the model.”
Efficient Bidirectional Neural Machine Translation (2019, background): “Neural Machine Translation (NMT) is very challenging and attracts lots of research attention in recent years (Cho et al., 2014; Bahdanau et al., 2015; Luong et al., 2015; Gehring et al., 2017a; Vaswani et al., 2017a; Shen et al., 2018; He et al., 2018; Hassan et al., 2018; Song et al., 2019).”
Neural Machine Translation: A Review (2019, background): “[445] does not only use the last encoder layer as hidden representation, but instead connects encoder and decoder layers at the same depth via attention.”
The TALP-UPC Machine Translation Systems for WMT19 News Translation Task: Pivoting Techniques for Low Resource MT (2019, methodology): “5.3 Joint Source-Target Self-Attention NMT The translation system trained on the augmented Kazakh-English data and used for the ﬁnal WMT submissions is based on the architecture proposed by (He et al., 2018; Fonollosa et al., 2019).”
Sentence-wise Smooth Regularization for Sequence to Sequence Learning (2018, background): “d great success in many natural language processing (NLP) tasks such as machine translation (Bahdanau, Cho, and Bengio 2014; Cho et al. 2014; Wu et al. 2016; Vaswani et al. 2017; Gehring et al. 2017; He et al. 2018; Song et al. 2018; Shen et al. 2018), text summarization (Rush et al. 2015; Shen et a”
Alternating Language Modeling for Cross-Lingual Pre-Training (2020, background): “…(Vaswani et al. 2017) 28.40 ConvS2S (Gehring et al. 2017) 25.16 Weighted Transformer (Ahmed, Keskar, and Socher 2017) 28.90 Layer-wise Transformer (He et al. 2018) 29.01 RNMT+ (Chen et 1 and Table 2 show that our ALM has signiﬁcant improvements over baselines without pre-training or with…”
From Bilingual to Multilingual Neural Machine Translation by Incremental Training (2019, methodology): “This information could be used in the future to train encoders of new languages by wisely sharing parameters with the decoder as in previous works (He et al., 2018).”
Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input (2018, background): “…machine translation, with different choices of architectures such as recurrent neural networks (RNNs) (Bahdanau, Cho, and Bengio 2014; Cho et al. 2014b), convolutional neural networks (CNNs) (Gehring et al. 2017), as well as self-attention based trans-former (Vaswani et al. 2017; He et al. 2018).”
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (2023, methodology): “When testing, it predicts the target sequence after inputting the source sequence into the LM. LM-SPE adopts the Separate Positional Encoding (Conneau and Lample, 2019; He et al., 2018) in an LM as the positional encoding.”
Syntax-Aware Data Augmentation for Neural Machine Translation (2020, background): “[18] shared the parameters of each layer between the encoder and decoder”
A Study of Multilingual Neural Machine Translation (2019, background): “…recent years (Bahdanau, Cho, and Bengio 2015; Luong, Pham, and Manning 2015; Sutskever, Vinyals, and Le 2014; Wu et al. 2016; Gehring et al. 2017; Shen et al. 2018; Vaswani et al. 2017; He et al. 2018; Hassan et al. 2018) for single-pair translation with a large number of bilingual sentence pairs.”
OmniNet: Omnidirectional Representations from Transformers (2021, background): “(He et al., 2018) proposed layer-wise coordination between encoder-decoder for machine translation.”