Story Cloze Task: UW NLP System

By Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, Noah A. Smith. 2017

48 citations recorded by the source corpus.

This paper describes University of Washington NLP’s submission for the Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem 2017) shared task—the Story Cloze Task. Our system is a linear classifier with a variety of features, including both the scores of a neural language model and style features. We report 75.2% accuracy on the task. A further discussion of our results can be found in Schwartz et al. (2017).

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Hypothesis Only Baselines in Natural Language Inference (2018, background): “It has also been shown that features focusing on sentence length, sentiment, and negation are sufﬁcient for achieving high accuracy on this dataset (Schwartz et al., 2017a; Cai et al., 2017; Bugert et al., 2017).”
The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task (2017, background): “…style is affected by the nature of the writing task the writer was asked to perform, 1This paper extends our LSDSem 2017 shared task submission (Schwartz et al., 2017). since different tasks likely engage different cognitive processes (Campbell and Pennebaker, 2003; Banerjee et al., 2014).2 We…”
Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task (2017, methodology): “(2016) and approaching the state of the art results of concurrent work (Schwartz et al., 2017b).”
What Makes Reading Comprehension Questions Easier? (2018, background): “A recent attempt at this task showed that recognizing superﬁcial features in the correct candidate is critical to achieve the state of the art (Schwartz et al., 2017).”
On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference (2019, background): “ing tasks (Schwartz et al., 2017), including visual question answering, where recent work has considered similar adversarial techniques for removing language biases (Ramakrishnan et al.”
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning (2020, background): “Human-annotated datasets usually contain biases (Schwartz et al., 2017; Cai et al., 2017; Bugert et al., 2017; Poliak et al., 2018; Gururangan et al., 2018; Zellers et al., 2019), which are often exploited by neural network models as shortcut solutions to achieve high testing accuracy.”
Story Comprehension for Predicting What Happens Next (2017, background): “Msap: The task addressed in this paper was also a shared task for an EACL’17 workshop and this baseline (Schwartz et al., 2017) represents the best performance reported on its leader-”
Tackling the Story Ending Biases in The Story Cloze Test (2018, methodology): “Furthermore, we conducted an extensive n-gram analysis, using word tokens, characters, part-of-speech, and token-POS (similar to Schwartz et al. (Schwartz et al., 2017b)) as features.”
Incorporating Structured Commonsense Knowledge in Story Completion (2018, methodology): “Story Completion Traditional machine learning meth-ods have been used to solve ROCStory Cloze Task such as (Schwartz et al. 2017).”
Language Models are Unsupervised Multitask Learners (2019, background): “When only minimal or no supervised data is available, another line of work has demonstrated the promise of language models to perform specific tasks, such as commonsense reasoning (Schwartz et al., 2017) and sentiment analysis (Radford et al.”
Story Ending Selection by Finding Hints From Pairwise Candidate Endings (2019, methodology): “Traditional feature-basedmethods for story understanding [18]– [21] adopted some shallow semantic features, such as n-grams and POS tags and trained a linear regression model to determine whether a candidate ending is plausible.”