CharacTer: Translation Edit Rate on Character Level

By Weiyue Wang, Jan-Thorsten Peter, Hendrik Rosendahl, Hermann Ney. 2016

156 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “N-gram based metrics (Papineni et al., 2002; Doddington, 2002; Wang et al., 2016; Popović, 2015) that measure lexical overlap require tokenization which is highly affected by language specific factors such as alphabet size and morphological complexity.”
Results of the WMT16 Metrics Shared Task (2016, background): “Metric Participant BEER ILLC – UvA (Stanojević and Sima’an, 2015) CharacTer RWTH Aachen University (Wang et al., 2016) chrF1,2,3 Humboldt University of Berlin (Popović, 2016) wordF1,2,3 Humboldt University of Berlin (Popović, 2016) DepCheck Charles University, no corresponding paper DPMFcomb- Chines”
Results of the WMT17 Metrics Shared Task (2017, methodology): “C HARAC T ER (Wang et al., 2016), identical to the 2016 setup, is a character-level metric inspired by the commonly applied translation edit rate (TER).”
Evaluation of Text Generation: A Survey (2020, background): “Two recent variants, CharacTER (Wang et al., 2016), a character-based translation edit distance measure, and EED (Stanchev et al., 2019), an extension of Levenshtein distance, have shown to correlate better with human judgments on some languages.”
Evaluating the morphological competence of Machine Translation Systems (2017, methodology): “BLEU to detect improvements targeting speciﬁc difﬁculties of MT, several problem-speciﬁc measures have been introduced over the years such as the LR-Score (Birch and Osborne, 2010) to measure the correctness of reordering decisions, MEANT (Lo and Wu, 2011) to measure the transfer of entailment relat”
RUSE: Regressor Using Sentence Embeddings for Automatic Machine Translation Evaluation (2018, methodology): “It incorporates 25 lexical metrics provided by the Asiya MT evaluation toolkit, as well as four other metrics, namely BEER (Stanojević and Sima’an, 2015), CharacTER (Wang et al., 2016), DPMF, and ENTFp.”
Results of the WMT20 Metrics Shared Task (2020, methodology): “Similarly to other characterlevel metrics, CHARACTER is generally applied to nontokenized outputs and references, which also holds for this year’s submission with one exception.”
FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN (2020, background): “, 2006a), BEER (Stanojevic and Sima'an, 2014), and CharacTER (Wang et al., 2016).”
Tilde’s Machine Translation Systems for WMT 2019 (2019, methodology): “We evaluated all MT systems using multiple automatic evaluation metrics including BLEU (Papineni et al., 2002), BEER 2.0 (Stanojevic and Sima’an, 2014), CharacTER (Wang et al., 2016), and TER (Snover et al., 2006).”
Open Source Toolkit for Speech to Text Translation (2018, methodology): “, 2006), the BEER metric (Stanojevic and Sima’an, 2014) and CharacTER (Wang et al., 2016).”
Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning (2017, methodology): “The machine translation output is evaluated with BLEU (Papineni et al., 2002), BEER (Stanojevic and Sima’an, 2014) and CharacTER (Wang et al., 2016).”
A Survey of Evaluation Metrics Used for NLG Systems (2020, background): “The main motivation for using character-based metrics is their improved performance in evaluating morphologically rich languages [126, 165].”
WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation (2019, background): “Character-based metrics like chrF (Popovic, 2015) and CharacTER (Wang et al., 2016) also re-lax the exact word match constraint by allowing the matching of characters.”