Facts2Story: Controlling Text Generation by Key Facts

By Eyal Orbach, Yoav Goldberg. 2020

16 citations recorded by the source corpus.

Recent advancements in self-attention neural network architectures have raised the bar for open-ended text generation. Yet, while current methods are capable of producing a coherent text which is several hundred words long, attaining control over the content that is being generated—as well as evaluating it—are still open questions. We propose a controlled generation task which is based on expanding a sequence of facts, expressed in natural language, into a longer narrative. We introduce human-based evaluation metrics for this task, as well as a method for deriving a large training dataset. We evaluate three methods on this task, based on fine-tuning pre-trained models. We show that while auto-regressive, unidirectional Language Models such as GPT2 produce better fluency, they struggle to adhere to the requested facts. We propose a plan-and-cloze model (using fine-tuned XLNet) which produces competitive fluency while adhering to the requested content.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Natural Language Generation

Citations in the corpus

• LongForm: Effective Instruction Tuning with Reverse Instructions (2023, neutral): “…ten tasks including poem generation (Hipson and Mohammad, 2020), table to text generation (Iyyer et al., 2017), story generation (Sap et al., 2020; Orbach and Goldberg, 2020; Lin et al., 2019), summarization (Fabbri et al., 2019; Kornilova and Eidelman, 2019; Bražin-skas et al., 2020), and fact…”