Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer

By Sudha Rao, Joel R. Tetreault. 2018

442 citations recorded by the source corpus.

Style transfer is the task of automatically transforming a piece of text in one particular style into another. A major barrier to progress in this field has been a lack of training and evaluation datasets, as well as benchmarks and automatic metrics. In this work, we create the largest corpus for a particular stylistic transfer (formality) and show that techniques from the machine translation community can serve as strong baselines for future work. We also discuss challenges of using automatic metrics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

A Recipe for Arbitrary Text Style Transfer with Large Language Models (2021, background): “Although style transfer has garnered increased interest due to the success of deep learning, these approaches usually require a substantial amount of labeled training examples, either as parallel text data (Zhu et al., 2010; Rao and Tetreault, 2018) or non-parallel text data of a single style.”
Controlling Style in Generated Dialogue (2020, neutral): “Bansal, 2018), formal (Rao and Tetreault, 2018), empathetic (Rashkin et al.”
Politeness Transfer: A Tag and Generate Approach (2020, background): “Prior work on text style transfer (Shen et al., 2017; Li et al., 2018; Prabhumoye et al., 2018; Rao and Tetreault, 2018; Xu et al., 2012; Jham-tani et al., 2017) has not focused on politeness as a style transfer task, and we argue that deﬁning it is cumbersome.”
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020, methodology): “To achieve the desired character portrayal, a system must be able to change the underlying meaning of events, unlike certain formalizations (e.g., polite-ness transfer; Rao and Tetreault, 2018) where full meaning preservation is required.”
Fair Transfer of Multiple Style Attributes in Text (2019, methodology): “On the other hand, the Formality dataset was part of the Yahoo Answers corpus L6, which was labeled in [12].”
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters (2023, methodology): “from the Transformers Library that is fine-tuned on Grammarly’s Yahoo Answers Formality Corpus (GYAFC) (Rao and Tetreault, 2018).”
Answer-based Adversarial Training for Generating Clarification Questions (2019, background): “…(Bahdanau et al., 2015; Luong et al., 2015), summarization (Nallapati et al., 2016), dialog (Bor-des et al., 2016; Li et al., 2016a; Serban et al., 2017), textual style transfer (Jhamtani et al., 2017; Rao and Tetreault, 2018) and question answering (Yin et al., 2016; Serban et al., 2016).”
A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer (2019, methodology): “A newly released dataset G YAFC (Grammarly’s Yahoo Answers Formality Corpus ) [ Rao and Tetreault, 2018 ] is used for this task.”
StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer (2021, background): “, 2017) and GYAFC for formality (Rao and Tetreault, 2018)) can in fact be seen as composed from a dictionary of fine-grained style constructs.”
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models (2022, methodology): “For sentiment (Yelp) and formality (GYAFC) transfer where we have reference text, we report the BLEU score.”
Controlled Text Generation as Continuous Optimization with Multiple Constraints (2021, methodology): “(Entertainment/Music domain; around 50K formal sentences) [53] and evaluate it on the provided test set containing 1312 informal sentences.”
Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting (2019, methodology): “Following previous work (Rao and Tetreault, 2018), we assess the model output on three criteria: formality, fluency and meaning preservation.”
IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation (2019, background): “F ORMALITY The F ORMALITY dataset stems from an aligned set of formal and informal sentences (Rao and Tetreault, 2018).”
Civil Rephrases Of Toxic Texts With Self-Supervised Transformers (2021, background): “…al., 2011; He and McAuley, 2016; Shen et al., 2017; Li et al., 2018), but also addresses sociolinguistic datasets containing text in various registers (Gan et al., 2017; Rao and Tetreault, 2018) or with different identity markers (Voigt et al., 2018; Prabhu-moye et al., 2018; Lample et al., 2019).”
(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personas (2019, background): “…works transfer style between text on single type of style aspect such as sentiment (Fu et al., 2018; Shen et al., 2017; Hu et al., 2017), gender (Reddy and Knight, 2016), political orientation (Prab-humoye et al., 2018), and two conﬂicting corpora (e.g, paper and news (Han et al., 2017), or…”
Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020, background): “Our second dataset is Formality transfer (Rao and Tetreault, 2018), which contains 105k sentences, also from two styles.”