Sentence-level MT evaluation without reference translations: beyond language modeling
By Michael Gamon, Anthony Aue, Martine Smets. 2005
144 citations recorded by the source corpus.
Subfields: Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
In Evaluation & Benchmarks
Citations in the corpus
- How Can We Know What Language Models Know? (2019, methodology): “…most probable token conditioned on the other tokens, inspired by the mask-predict decoding algorithm used in non-autoregressive machine translation (Ghazvininejad et al., 2019):9 PLM(wi|tr \ i) = ∑ 〈x,y〉∈R PLM(wi|x, tr \ i, y) |R| , where wi is the i-th token in the prompt and tr \ i is the prompt…”
- Machine Translation Detection from Monolingual Web-Text (2013, background): “…judgment in terms of the quality of machine-translated sentences based on features extracted using a syntactic parser (Corston-Oliver et al., 2001; Gamon et al., 2005; Avramidis et al., 2011) or essay scoring system (Parton et al., 2011), assuming that their input is always machine-translated.”
- Improved Natural Language Generation via Loss Truncation (2020, background): “Instead of optimizing log loss, which has little correlation with model output quality (Theis et al., 2016; Hashimoto et al., 2019; Gamon et al., 2005), recent work on diverse generation models has proposed optimizing for the distinguishability of samples from the model and the reference.”
- Predicting Grammaticality on an Ordinal Scale (2014, background): “Regarding sentence-level grammaticality, there has been much work on rating the grammaticality of machine translation outputs (Gamon et al., 2005; Parton et al., 2011), such as the MT Quality Estimation Shared Tasks (Bojar et al., 2013, § 6), but relatively little on evaluating the grammaticality…”
- Quality estimation for translation selection (2014, background): “With similar classifiers (albeit different datasets), Gamon et al. [2005] re-ported as trivial the problem of distinguishing human translations from machine translations.”
- Cross-Language Document Summarization Based on Machine Translation Quality Prediction (2010, background): “Gamon et al. (2005) investigate the possibility of evaluating MT quality and fluency at the sentence level in the absence of reference translations, and they can improve on the correlation between language model perplexity scores and human judgment by combing these perplexity scores with class…”