Understanding and Improving Morphological Learning in the Neural Machine Translation Decoder

By Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Stephan Vogel. 2017

60 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives (2019, background): “Different levels of linguistic analysis have been considered including morphology (Belinkov et al., 2017a; Dalvi et al., 2017; Bisazza and Tump, 2018), syntax (Shi et al.”
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned (2019, background): “Different levels of linguistic analysis have been considered including morphology (Belinkov et al., 2017a; Dalvi et al., 2017; Bisazza and Tump, 2018), syntax (Shi et al.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “, 2017b) NMT states POS, lexical semantics Classification (Belinkov et al., 2017a; Dalvi et al., 2017) NMT states POS, morphology Classification”
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (2018, background): “It is not clear that they controlled for the same factors we considered (in particular, lexical overlap and Dalvi et al. (2017) are also interested in understanding the type of linguistic knowledge encoded in sentence and word embeddings, but their focus is on word-level morphosyntax and lexical…”
Synthetic and Natural Noise Both Break Neural Machine Translation (2017, background): “Several studies investigated the ability of such models to learn linguistic properties at morphological (Vylomova et al., 2016; Belinkov et al., 2017a; Dalvi et al., 2017), syntactic (Shi et al.”
The Lazy Encoder: A Fine-Grained Analysis of the Role of Morphology in Neural Machine Translation (2018, neutral): “While this methodology is similar to that of previous work (K¨ohn, 2015; Belinkov et al., 2017a,b; Dalvi et al., 2017) we make sure that our results are not affected by overﬁtting by eliminating any vocabulary overlap between the classiﬁer’s training and test sets.”
Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition (2019, background): “Others focused on intermediate representations in neural machine translation systems (e.g., Shi et al., 2016; Belinkov et al., 2017; Dalvi et al., 2017; Sennrich, 2017), or on specific linguistic properties such as agreement (Giulianelli et al., 2018), and tense (Bacon and Regier, 2018).”
On the Linguistic Representational Power of Neural Machine Translation Models (2019, background): “The language properties mainly analyzed are morphological (Qian, Qiu, and Huang 2016b; Vylomova et al. 2016; Belinkov et al. 2017a; Dalvi et al. 2017), semantic (Qian, Qiu, and Huang 2016b; Belinkov et al.”
No Training Required: Exploring Random Encoders for Sentence Classification (2019, background): “…what linguistic knowledge is encoded in word and sentence embeddings, for instance in machine translation (Belinkov et al., 2017; Sen-nrich, 2016; Dalvi et al., 2017), with a focus on evaluating RNNs or LSTMs (Linzen et al., 2016; Hupkes et al., 2018) or even sequence-to-sequence models (Lake &…”
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation (2018, background): “In addition to syntax (Shi et al., 2016), researchers have used other labeling tasks to investigate whether neural machine translation (NMT) models learn different linguistic phenomena (Belinkov et al., 2017a,b; Dalvi et al., 2017; Marvin and Koehn, 2018).”
On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference (2018, background): “Researchers have explored what NMT models learn about other linguistic phenomena, such as morphology (Dalvi et al., 2017; Belinkov et al., 2017a), syntax (Shi et al., 2016), and lexical semantics (Belinkov et al., 2017b), including word senses (Marvin and Koehn, 2018; Liu et al., 2018)”
Towards a Unified Natural Language Inference Framework to Evaluate Sentence Representations (2018, methodology): “, 2016), researchers have used other labeling tasks to investigate whether neural machine translation (NMT) models learn other types of linguistic phenomena (Belinkov et al., 2017b,a; Dalvi et al., 2017; Marvin and Koehn, 2018).”
Analyzing Individual Neurons in Pre-trained Language Models (2020, unknown): “…such as machine translation and language modeling, these deep models capture fundamental linguistic phenomena such as word morphology, syntax and various other relevant properties of interest (Shi et al., 2016; Adi et al., 2016; Belinkov et al., 2017a,b; Dalvi et al., 2017; Blevins et al., 2018).”
Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation (2018, background): “Various recent works have attempted to improve NMT by scaffolding translation task on a single auxiliary task (Domhan and Hieber, 2017; Zhang and Zong, 2016; Dalvi et al., 2017).”
Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection (2019, background): “Grönroos et al. (2017) showed that incorporating morphological analysis in the decoder improves NMT performance for morphologically rich languages.”
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (2019, methodology): “…2016; Wang et al., 2017), and NMT representations (Shi et al., 2016; Belinkov et al., 2017a), as applied to morphological (Vy-lomova et al., 2017; Dalvi et al., 2017), semantic (Qian et al., 2016b; Belinkov et al., 2017b) and syntactic (Linzen et al., 2016; Tran et al., 2018; Conneau et al.,…”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “Chapter 2 is mainly based on [32], with additional experiments from [83].”
What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models (2018, background): “…analyzing word and sentence embeddings (Adi et al. 2016; Qian, Qiu, and Huang 2016b; Conneau et al. 2018), recurrent neural network (RNN) states (Shi, Padhi, and Knight 2016; Wang, Chung, and Lee 2017), and NMT representations (Belinkov et al. 2017a; Belinkov et al. 2017b; Dalvi et al. 2017).”