OpenNMT: Open-Source Toolkit for Neural Machine Translation

By Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, Alexander M. Rush. 2017

1,966 citations recorded by the source corpus.

We describe an open-source toolkit for neural machine translation (NMT). The toolkit prioritizes efficiency, modularity, and extensibility with the goal of supporting NMT research into model architectures, feature representations, and source modalities, while maintaining competitive performance and reasonable training requirements. The toolkit consists of modeling and translation support, as well as detailed pedagogical documentation about the underlying techniques.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Massive Exploration of Neural Machine Translation Architectures (2017, background): “Our model is outper-formed only by (Wu et al., 2016 (Jean et al., 2015), RNNSearch-LV (Jean et al., 2015), BPE (Sennrich et al., 2016b), BPE-Char (Chung et al., 2016), Deep-Att (Zhou et al., 2016), Luong (Lu-ong et al., 2015a), Deep-Conv (Gehring et al., 2016), GNMT (Wu et al., 2016), and OpenNMT (K”
Dissecting Contextual Word Embeddings: Architecture and Representation (2018, methodology): “…for bidirectional language modeling, we modiﬁed a PyTorch based 1 http://allennlp.org/elmo 2 https://github.com/allenai/bilm-tf re-implementation (Klein et al., 2017) 3 to mask out future tokens for the forward language model and previous tokens for the backward language model, in a similar…”
Learned in Translation: Contextualized Word Vectors (2017, methodology): “We begin by training an attentional sequence-to-sequence model for English-to-German translation based on Klein et al. [2017] with the goal of transferring the encoder to other tasks.”
CoQA: A Conversational Question Answering Challenge (2018, methodology): “For all the experiments of seq2seq and PGNet, we use the OpenNMT toolkit (Klein et al., 2017) and its default settings: 2-layers of LSTMs with 500 hidden units for both the encoder and the decoder.”
Deep Learning Scaling is Predictable, Empirically (2017, methodology): “We use a publicly available implementation of this architecture in OpenNMT (Klein et al. (2017)). The encoder contains two layers of bidirectional LSTMs, and the decoder contains the attention layer and stack of LSTM layers. To simplify training this SOTA model, we remove ensembling and data augment”
Unified Language Model Pre-training for Natural Language Understanding and Generation (2019, methodology): “Both Transformer [43] and OpenNMT [23] implement standard attentional sequence-to-sequence models.”
Fine-tune BERT for Extractive Summarization (2019, methodology): “We use PyTorch, OpenNMT (Klein et al., 2017) and the ‘bert-base-uncased’ 1 version of BERT to implement the model.”
fairseq: A Fast, Extensible Toolkit for Sequence Modeling (2019, background): “For example, OpenNMT (Klein et al., 2017) is a community-built toolkit written in multiple languages with an emphasis on extensibility.”
MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers (2021, methodology): “…have been a number of toolkits that managed to summarize the existing methods and establish a uniﬁed framework for a certain task, such as OpenNMT (Klein et al., 2017) and TexBox (Li et al., 2021) for text generation tasks, ExplainaBoard (Liu et al., 2021) for evaluating interpretable models,…”
Learning Deep Transformer Models for Machine Translation (2019, methodology): “In recent implementations (Klein et al., 2017; Vaswani et al., 2018; Domhan, 2018), layer normalization is applied to the input of every sub-layer (see Figure 1(b)): Eq.”
Emerging Properties in Self-Supervised Vision Transformers (2021, neutral): “The set of patch tokens and [CLS] token are fed to a standard Transformer network with a “pre-norm” layer normalization [10, 32].”
Neural Text Generation with Unlikelihood Training (2019, background): “Separately, hard or soft beam blocking has been investigated (Klein et al., 2017), whereby previously generated n -grams should not be generated again.”
The WebNLG Challenge: Generating Text from RDF Data (2017, methodology): “On this delexicalised data-to-text corpus, we trained a vanilla sequence-to-sequence model with attention mechanism using the OpenNMT toolkit (Klein et al., 2017) with default parameters for training and decoding.”