Expectation-based syntactic comprehension.

By Roger Levy. Cognition, 2008

2,115 citations recorded by the source corpus.

This paper investigates the role of resource allocation as a source of processing difficulty in human sentence comprehension. The paper proposes a simple information-theoretic characterization of processing difficulty as the work incurred by resource reallocation during parallel, incremental, probabilistic disambiguation in sentence comprehension, and demonstrates its equivalence to the theory of Hale [Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL (Vol. 2, pp. 159-166)], in which the difficulty of a word is proportional to its surprisal (its negative log-probability) in the context within which it appears. This proposal subsumes and clarifies findings that high-constraint contexts can facilitate lexical processing, and connects these findings to well-known models of parallel constraint-based comprehension. In addition, the theory leads to a number of specific predictions about the role of expectation in syntactic comprehension, including the reversal of locality-based difficulty patterns in syntactically constrained contexts, and conditions under which increased ambiguity facilitates processing. The paper examines a range of established results bearing on these predictions, and shows that they are largely consistent with the surprisal theory.

Subfields: Artificial Intelligence, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/2a92864d3f44e1b34da0fa17e13d5752d089eddf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

Citations in the corpus

Understanding heritage languages (2019, background): “…low-frequency or unexpected material takes longer to process than predicted material; we know this from studies of surprisal (e.g., Hale, 2001; Levy, 2008) and garden-path effects (e.g., Christianson, Hollingworth, Halliwell & Ferreira, 2001; Ferreira & Henderson, 1991; Frazier & Rayner,…”
Coherence and Coreference Revisited (2007, neutral): “bilistic, expectation-driven models (e.g., Hale 2001; Levy 2007).”
Rapid Expectation Adaptation during Syntactic Comprehension (2013, background): “We build on and attempt to synthesize insights from three research traditions that have up until now proceeded largely in parallel: experience- or expectation-based theories of language comprehension [11,18,19], syntactic priming in comprehension [20–22], and psycholinguistic research exploring the ”
Incrementality and Prediction in Human Sentence Processing (2009, background): “…nor are our observations particularly unique; there is a growing body of computational work under the banner of expectation-based parsing, drawing on insights from information theory, which is predicated on parsing as a probabilistic and inherently predictive process (e.g., Hale, 2001; Levy, 2008).”
The neural architecture of language: Integrative modeling converges on predictive processing (2020, background): “…McRae et al., 1998; Rohde & Plaut, 1999; Spivey & Tanenhaus, 1998; Tenenbaum et al., 2011) and neuroscience (Bastos et al., 2012; Keller & Mrsic-Flogel, 2018; Mumford, 1992; Rao & Ballard, 1999; Srinivasan et al., 1982), including in the domain of language (Kuperberg & Jaeger, 2016; Levy, 2008a).”
Music, biological evolution, and the brain (2010, background): “To take another example, in both domains incoming categories vary in how easy they are to integrate into the existing structural representation of the sequence (Bigand et al., 2003; Koelsch et al., 2007; Gibson, 1998; Levy, 2008).”
Redundancy and reduction: Speakers manage syntactic information density (2010, background): “This links the results reported here to findings suggesting that syntactic comprehension is probability-sensitive (Hale, 2001; Hale, 2003; Jurafsky, 1996; Kamide et al., 2003; Levy, 2008; MacDonald, 1994; McDonald & Shillcock, 2003; Staub & Clifton, 2006; Trueswell, 1996).”
Modeling the effects of memory on human online sentence processing with particle filters (2008, background): “An important feature distinguishing “rational” models of online sentence comprehension [6, 7, 8, 9] from what are sometimes called “dynamical systems” models [25, 15] is that the latter have an internal feedback mechanism: in the absence of any biasing input, the activation of the leading candidate ”
Semantic reconstruction of continuous language from non-invasive brain recordings (2022, methodology): “The word-level match scores were also compared to word duration in the test dataset, language model probability in the test data-set (which corresponds to the information conveyed by a word) 71 , word frequency in the test dataset and word frequency in the training dataset.”
RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency (2018, background): “These reading times are often taken to reﬂect the extent to which humans expect certain words in context, and may be generally proportional to surprisal given the com-prehender’s probabilistic language model (Hale, 2001; Levy, 2008; Smith and Levy, 2013).”
Zipf’s word frequency law in natural language: A critical review and future directions (2014, background): “…linked to theories of what human language comprehenders find predictable (e.g. Demberg & Keller, 2008; A. Frank & Jaeger, 2008; Jaeger, 2010; Levy, 2008; Levy & Jaeger, 2007; Piantadosi, Tily, & Gibson, 2011; N. J. Smith & Levy, in press) and how much information is effectively conveyed for…”
PSYCHOLOGICAL SCIENCE Research Article Using Speakers ’ Referential Intentions to Model Early Cross-Situational Word Learning (positive): “In order to link the probability of a particular state of the world under our model to the looking time of infants in Xu’s study, we used surprisal (negative log probability), a measure that has previously been used successfully to link model probabilities to human reaction time data (Levy, 2007).”
Plausibility and recovery from garden paths in second language sentence processing (2011, background): “As stated earlier, models of sentence comprehension differ in how reanalysis is handled (e.g., Gibson, 1991; Hale, 2003; Levy, 2008; Spivey & Tanenhaus, 1998; Trueswell & Kim, 1998) but all assume that weak and strong GPs would elicit different processing costs.”
More than words: Frequency effects for multi-word phrases (2010, neutral): “Such models include (but are not limited to) constraint-satisfaction and expectation-based models of comprehension (Hale, 2001; Jurafsky, 1996, 2003; Levy, 2008; MacDonald, 1994; MacDonald et al., 1994; McRae, Spivey-Knowlton, & Tanenhaus, 1998; Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995).”