The representational geometry of word meanings acquired by neural machine translation models

43 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Learned in Translation: Contextualized Word Vectors (2017, positive): “Hill et al. [2017] further demonstrate that ﬁxed-length representations obtained from NMT encoders outperform those obtained from monolingual (e.g. language modeling) encoders on semantic similarity tasks.”
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives (2019, methodology): “It is straightforward to apply the Transformer to this task (Radford et al., 2018; Lample and Conneau, 2019).”
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned (2019, background): “Different levels of linguistic analysis have been considered including morphology (Belinkov et al., 2017a; Dalvi et al., 2017; Bisazza and Tump, 2018), syntax (Shi et al., 2016) and semantics (Hill et al., 2017; Belinkov et al., 2017b; Raganato and Tiedemann, 2018).”
Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation (2018, background): “In Hill et al. (2017), the authors show the effectiveness of representations obtained from an NMT model in semantic similarity tasks.”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “[148] analyzed word embeddings in neural machine translation models and found that they outperform monolingual word embeddings on semantic similarity tasks.”