Neural CRF Model for Sentence Alignment in Text Simplification

By Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, Wei Xu. 2020

187 citations recorded by the source corpus.

The success of a text simplification system heavily depends on the quality and quantity of complex-simple sentence pairs in the training corpus, which are extracted by aligning sentences between parallel articles. To evaluate and improve sentence alignment quality, we create two manually annotated sentence-aligned datasets from two commonly used text simplification corpora, Newsela and Wikipedia. We propose a novel neural CRF alignment model which not only leverages the sequential nature of sentences in parallel documents but also utilizes a neural sentence pair model to capture semantic similarity. Experiments demonstrate that our proposed approach outperforms all the previous work on monolingual sentence alignment task by more than 5 points in F1. We apply our CRF aligner to construct two new text simplification datasets, Newsela-Auto and Wiki-Auto, which are much larger and of better quality compared to the existing datasets. A Transformer-based seq2seq model trained on our datasets establishes a new state-of-the-art for text simplification in both automatic and human evaluation.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, background): “WikiAuto + Turk/ASSET (Jiang et al., 2020; Xu et al., 2016; Alva-Manchego et al., 2020) 30.”
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, neutral): “…et al. 2020 tweet_qa qa/machine reading comprehension Xiong et al. 2019 web_questions qa/closed-book qa Berant et al. 2013 wiki_auto cls/other Jiang et al. 2020 wiki_bio cg/other Lebret et al. 2016 wiki_qa cls/other Yang et al. 2015 wiki_split cg/other Botha et al. 2018 wikisql cg/other…”
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer (2021, background): “…have since focused on developing prompt-based learning approaches with carefully handcrafted prompts (Schick and Schütze, 2021), prompt mining and paraphrasing (Jiang et al., 2020b), gradient-based search for improved prompts (Shin et al., 2020), and automatic prompt generation (Gao et al., 2021).”
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models (2022, methodology): “Jiang et al. (2020) constructs a corpus consisting of 677K complexsimple sentences with revision alignment.”
Fine-tuned Language Models are Continual Learners (2022, methodology): “These tasks include: 1) Text Sim-plification (Simpl) with the goal of paraphrasing a given text using simple language, where we train our model on WikiAuto Jiang et al. (2020) and evaluate it on the WikiAuto and ASSET datasets (Alva-Manchego et al., 2020); 2) Headline Generation with Constraint (HGe”
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022, background): “Three of the datasets are simpliﬁcation evaluation sets added to the WikiAuto loader (Jiang et al., 2020), while all others have independent data loaders.”
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases (2022, unknown): “We experimented with other alignments (wiki-auto and newsela-auto (Jiang et al., 2020)) but with lower performance.”
Continual-T0: Progressively Instructing 50+ Tasks to Language Models Without Forgetting (2022, background): “Text Simpliﬁcation (Simpl) Jiang et al. (2020) provided WikiAuto, a set of 400,000 aligned sentences from English Wikipedia and Simple English Wikipedia as a resource to train sentence simpliﬁcation systems.”
The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation (2021, unknown): “Each paper was published between 2018 and 2020 at ACL, NAACL, or EMNLP, and we exclude papers that use AMT to evaluate more wellestablished generation tasks like machine translation, or summarization.3 Unlike previous surveys of evaluating generated text (Çelikyilmaz et al., 2020; van der Lee et al.”