Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation
By Jesús Giménez, Lluís Màrquez i Villodre. 2010
102 citations recorded by the source corpus.
Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation This article describes the Asiya Toolkit for Automatic Machine Translation Evaluation and Meta-evaluation, an open framework offering system and metric developers a text interface to a rich repository of metrics and meta-metrics.
Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ondrej Bojar
- Philipp Koehn
- Lucia Specia
- Matt Post
- Graham Neubig
- Barry Haddow
- Rico Sennrich
- Felix Stahlberg
In Natural Language Processing
- Ondrej Bojar
- Philipp Koehn
- Lucia Specia
- Barry Haddow
- Matt Post
- Christof Monz
- Yvette Graham
- Graham Neubig
In Evaluation & Benchmarks
- Ondrej Bojar
- Lucia Specia
- Philipp Koehn
- Matt Post
- Yvette Graham
- Barry Haddow
- Christian Federmann
- Marco Turchi
Citations in the corpus
- Findings of the 2016 Conference on Machine Translation (2016, methodology): “HTER was calculated by using the Asiya toolkit implementation of TER (non-tokenised and case insensitive) (Giménez and Màrquez, 2010).”
- RUSE: Regressor Using Sentence Embeddings for Automatic Machine Translation Evaluation (2018, methodology): “It incorporates 55 default metrics provided by the Asiya MT evaluation toolkit2 (Giménez and Màrquez, 2010), as well as three other metrics, namely DPMF (Yu et al., 2015b), REDp (Yu et al., 2015a), and ENTFp (Yu et al., 2015a), using ranking SVM to train parameters of each metric score.”
- A Survey of Evaluation Metrics Used for NLG Systems (2020, methodology): “17 syntactic metrics are borrowed from the Asiya toolkit [52] along with 13 semantic metrics, which in reality correspond to 3 distinct metrics, related to Named entities, Semantic Roles and Discourse Representation.”