EASSE: Easier Automatic Sentence Simplification Evaluation

By Fernando Alva-Manchego, Louis Martin, Carolina Scarton, Lucia Specia. 2019

149 citations recorded by the source corpus.

We introduce EASSE, a Python package aiming to facilitate and standardise automatic evaluation and comparison of Sentence Simplification (SS) systems. EASSE provides a single access point to a broad range of evaluation resources: standard automatic metrics for assessing SS outputs (e.g. SARI), word-level accuracy scores for certain simplification transformations, reference-independent quality estimation features (e.g. compression ratio), and standard test data for SS evaluation (e.g. TurkCorpus). Finally, EASSE generates easy-to-visualise reports on the various metrics and features above and on how a particular SS output fares against reference simplifications. Through experiments, we show that these functionalities allow for better comparison and understanding of the performance of SS systems.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

In Natural Language Generation

Citations in the corpus

Control Prefixes for Text Generation (2021, methodology): “In addition to reporting the cus- 9We use the FKGL and latest version of SARI implemented in EASSE (Alva-Manchego et al., 2019) which is used in Martin et al. (2020). tomary ROUGE scores (Lin, 2004), we submit our CONTROL PREFIXES model outputs to the GENIE external human evaluation framework (Khash”
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases (2022, unknown): “SARI score computation We use the latest version of SARI implemented in EASSE (Alva-Manchego et al., 2019) which fixes bugs and inconsistencies from the traditional implementation of SARI.”
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations (2020, methodology): “In our experiments, we used the implementations of these metrics available in the EASSE package for automatic sentence simpliﬁcation evaluation (Alva-Manchego et al., 2019).”
Control Prefixes for Parameter-Efficient Text Generation (2021, methodology): “13We use the FKGL and the latest version of SARI implemented in EASSE (Alva-Manchego et al., 2019) which is also used by Martin et al. (2020).”