MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases

By Louis Martin, Angela Fan, Eric Villemonte de la Clergerie, Antoine Bordes, Benoît Sagot. 2022

124 citations recorded by the source corpus.

Progress in sentence simplification has been hindered by a lack of labeled parallel simplification data, particularly in languages other than English. We introduce MUSS, a Multilingual Unsupervised Sentence Simplification system that does not require labeled simplification data. MUSS uses a novel approach to sentence simplification that trains strong models using sentence-level paraphrase data instead of proper simplification data. These models leverage unsupervised pretraining and controllable generation mechanisms to flexibly adjust attributes such as length and lexical complexity at inference time. We further present a method to mine such paraphrase data in any language from Common Crawl using semantic sentence embeddings, thus removing the need for labeled data. We evaluate our approach on English, French, and Spanish simplification benchmarks and closely match or outperform the previous best supervised results, despite not using any labeled simplification data. We push the state of the art further by incorporating labeled simplification data.

Subfields: Artificial Intelligence, Natural Language Processing, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Smaranda Muresan
2. Tuhin Chakrabarty
3. Thomas Scialom
4. Bin Liu
5. Zixuan Ke

In Natural Language Processing

1. Smaranda Muresan
2. Tuhin Chakrabarty
3. Thomas Scialom

Citations in the corpus

• Fine-tuned Language Models are Continual Learners (2022, background): “Moreover, CT0 obtains a new state-of-the-art on the ASSET evaluation set, improving over MUSS (Martin et al., 2020): 85.”
• Continual-T0: Progressively Instructing 50+ Tasks to Language Models Without Forgetting (2022, background): “Moreover, CT0 obtains a new state-of-the-art on the ASSET evaluation set, improving over MUSS (Martin et al., 2020): 85.9 BLEU4 Vs 72.98 and 46.6 SARI Vs 44.15, and despite not using all the training data available.”