Conditioned Natural Language Generation using only Unconditioned Language Model: An Exploration

By Fan-Keng Sun, Cheng-I Lai. 2020

18 citations recorded by the source corpus.

Transformer-based language models have shown to be very powerful for natural language generation (NLG). However, text generation conditioned on some user inputs, such as topics or attributes, is non-trivial. Past approach relies on either modifying the original LM architecture, re-training the LM on corpora with attribute labels, or having separately trained `guidance models' to guide text generation in decoding. We argued that the above approaches are not necessary, and the original unconditioned LM is sufficient for conditioned NLG. We evaluated our approaches by the samples' fluency and diversity with automated and human evaluation.

Subfields: Artificial Intelligence, Large Language Models, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Natural Language Generation

Citations in the corpus

"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters (2023, background): “Prompting (Brown et al., 2020; Sun and Lai, 2020) steers pre-trained language models with taskspecific instructions to generate task outputs without task fine-tuning.”
Can Foundation Models Wrangle Your Data? (2022, background): “Although recent work has tried to systematize the prompt tuning process, it is still an open area of research [55, 91].”
PADA: A Prompt-based Autoregressive Approach for Adaptation to Unseen Domains (2021, methodology): “…different purposes, yet all use it as a way for priming the model for in-context learning (Brown et al., 2020), to elicit context-speciﬁc information from the language model (Jiang et al., 2019; Sun and Lai, 2020; Shin et al., 2020), or as a method for tuning the model (Li and Liang, 2021).”