Neural Machine Translation

By Philipp Koehn. International Journal for Research in Applied Science and Engineering Technology, 2020

657 citations recorded by the source corpus.

Abstract: The project's novelty is not merely importing modules and preparing data and feeding the data to the model but understanding how the real language translation works and implementing the logics underlying each method utilized and creating every function from scratch, resulting in the creationof a Neural Machine Translation model. Initially, translation was accomplished by simply substituting words from one language for those from another. However, because languages are essentially different, a greater degree of knowledge (e.g., phrases/sentences) is required to achieveeffective results. With the introduction of deep learning, modern software now employs statisticaland neural techniques that have been shown to be more effective when translating. We are essentially translating German to English utilizing Sequence to Sequence models with attention and transformer models. Of course, everyone has access to Google Translates power, but if you want to learn how to implement translation in code, this project will show you how. We are writingour code from scratch, without using any libraries,in order to understand how each model works.While this design is a little out of date, it is still a great project to work on if you want to learn more about attention processes before moving on to Transformers. It is based on Effective Approaches to Attentionbased Neural Machine Translator and is a sequence to sequence (seq2seq) model for German to English translation.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/eea463ed69db876cdc12371d2bec200849b608a7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020, background): “, 2014; Kalchbrenner and Blunsom, 2013; Bahdanau et al., 2015; Vaswani et al., 2017), but are now used widely as a central component of a variety of NLP systems (e.g. Tan et al. (2017); Chollampatt and Ng (2018)).”
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021, background): “(Hochreiter and Schmidhuber, 1997; Bahdanau et al., 2015; Gu et al., 2016), fully attention-based Transformer architectures (Vaswani et al.”
NCNet: Deep Learning Network Models for Predicting Function of Non-coding DNA (2019, background): “…network approaches, another useful networks are recurrent neural networks (RNNs), which have been successfully applied with tremendous success (Graves et al., 2013; Bahdanau et al., 2014; Xu et al., 2015) such as in speech recognition, neural machine translation, and image caption generation.”
Referential translation machines for predicting semantic similarity (2015, methodology): “We also obtain results with lemmatized datasets using the Stanford POS tagger (Toutanova et al. 2003), where R uses the regular truecased (Koehn et al. 2007; Koehn 2010b) corpora, L uses the lemmatized truecased corpora, and R ?”
Uncertainty Estimation in Autoregressive Structured Prediction (2021, methodology): “Previous work has examined bespoke supervised confidence estimation techniques for each task separately (Evermann & Woodland, 2000; Liao & Gales, 2007; Ragni et al., 2018; Chen et al., 2017; Koehn, 2009; Kumar & Sarawagi, 2019) which construct an "error-detection" model on top of the original ASR/NM”
Investigating ‘Aspect’ in NMT and SMT: Translating the English Simple Past and Present Perfect (2017, background): “Neural Machine Translation (NMT) (Bahdanau et al. 2015, Cho et al. 2014, Sutskever et al. 2014) encodes the entire source sentence in an encoding vector.”
Neural Machine Translation: A Review (2019, background): “For even more insight into the field of neural machine translation, we refer the reader to other overview papers from Neubig (2017), Cromieres et al. (2017), Koehn (2017), PopescuBelis (2019).”
Transfer Learning for Low-Resource Neural Machine Translation (2016, neutral): “In the neural encoder-decoder framework for MT (Neco and Forcada, 1997; Castaño and Casacuberta, 1997; Sutskever et al., 2014; Bahdanau et al., 2014; Luong et al., 2015a), we use a recurrent neural network (encoder) to convert a source sentence into a dense, fixed-length vector.”