Strategies for Structuring Story Generation

By Angela Fan, Michael Lewis, Yann Dauphin. 2019

227 citations recorded by the source corpus.

Writers often rely on plans or sketches to write long stories, but most current language models generate word by word from left to right. We explore coarse-to-fine models for creating narrative texts of several hundred words, and introduce new models which decompose stories by abstracting over actions and entities. The model first generates the predicate-argument structure of the text, where different mentions of the same entity are marked with placeholder tokens. It then generates a surface realization of the predicate-argument structure, and finally replaces the entity placeholders with context-sensitive names and references. Human judges prefer the stories from our models to a wide range of previous approaches to hierarchical text generation. Extensive analysis shows that our methods can help improve the diversity and coherence of events and entities in generated stories.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

fairseq: A Fast, Extensible Toolkit for Sequence Modeling (2019, background): “…2017; Baevski and Auli, 2019), abstractive document summarization (Fan et al., 2018a; Liu et al., 2018; Narayan et al., 2018), story generation (Fan et al., 2018b, 2019), error correction (Chollampatt and Ng, 2018), multilingual sentence embeddings (Artetxe and Schwenk, 2018), and dialogue…”
Evaluation of Text Generation: A Survey (2020, methodology): “…Dong et al., 2019), using better sampling methods to reduce degeneration in decoding (Zellers et al., 2019; Holtzman et al., 2020), and learning to generate text with better discourse structures and narrative flow (Yao et al., 2018; Fan et al., 2019b; Dathathri et al., 2020; Rashkin et al., 2020).”
Controllable Story Generation with External Knowledge Using Large-Scale Language Models (2020, methodology): “We do not compare to Fan et al. (2019) because Guan et al. (2020) has already shown their model signiﬁcantly outperforms Fan et al. (2019) and in this work, we compare to Guan et al. (2020).”
Progressive Generation of Long Text with Pretrained Language Models (2021, methodology): “On the other hand, it is possible to adopt some of the content planning strategies (e.g., summaries or SRL sequences as the plans (Fan et al., 2019)), and repurpose pretrained LMs for generation in each stage.”
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View (2021, background): “…(Liu et al. 2017; Wiseman, Shieber, and Rush 2018; Shen et al. 2020; Li and Rush 2020; Ye et al. 2020) or controllable generation (Peng et al. 2018; Fan, Lewis, and Dauphin 2019; Shen et al. 2019; Parikh et al. 2020), which encourage the overlap between given structured data and generated text.”
Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals (2022, methodology): “Further metrics include counting the number of unique words, percentage of verbs and diversity in entity names [39], rare word usage and sentence length [102].”
STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation (2020, background): “…outputs using a combination of simple automatic metrics not designed for long-form creative text generation (e.g., BLEU and ROUGE against a single reference) and crowdsourced ratings (McIntyre and La-pata, 2009; Yao et al., 2019; Fan et al., 2019) that preclude evaluating long-form narratives.”
AggGen: Ordering and Aggregating while Generating (2021, methodology): “Similarly, Fan et al. (2019) use a pipeline approach for story generation via SRL-based sketches.”
The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation (2021, methodology): “You are the 10th... being hunted by the other 9...”), has been used in multiple previous works on paragraph-length story generation (Fan et al., 2019; See et al., 2019; Mao et al., 2019).”
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation (2020, background): “Another line is to decompose story generation into separate steps (Martin et al., 2018; Fan et al., 2018; Wang et al., 2016; Xu et al., 2018; Yao et al., 2019; Fan et al., 2019).”
InFillmore: Frame-Guided Language Generation with Bidirectional Context (2021, background): “…(Xu et al., 2018; Yao et al., 2019; Goldfarb-Tarrant et al., 2019; Gupta et al., 2019; Rashkin et al., 2020), short summaries (Jain et al., 2017; Chen et al., 2019), verb-argument tuples (Martin et al., 2018), or PropBank predicates and arguments (Fan et al., 2019; Goldfarb-Tarrant et al., 2020).”
Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning (2021, background): “…conditioning on content-relevant features such as plot outlines (Fan et al., 2018; Peng et al., 2018; Rashkin et al., 2020) or by hierarchical reasoning with abstract representations that help constrain story progression (Martin et al., 2018; Yao et al., 2019; Fan et al., 2019; Peng et al., 2021).”
Content Planning for Neural Story Generation with Aristotelian Rescoring (2020, methodology): “Existing work in neural story generation has established the strength of adding a content planning stage to structure the generated content (Yao et al., 2019; Fan et al., 2019) (discussed in more detail in Section 7).”
Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs (2021, unknown): “…al., 2017; Hu et al., 2017; Fu et al., 2018; Peng et al., 2018; Dai et al., 2019), tense (Logeswaran et al., 2018), plots (Ammanabrolu et al., 2020; Fan et al., 2019; Tambwekar et al., 2019; Yao et al., 2019; Goldfarb-Tarrant et al., 2019, 2020), societal bias (Wallace et al., 2019; Sheng et al.,…”