Systematicity in Connectionist Language Learning

By Robert F. Hadley. 1994

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Neural Logic Machines (2019, background): “Starting from Fodor & Pylyshyn (1988), however, there has been a debate over the problem of systematicity (such as understanding recursive systems) in connectionist models (Fodor & McLaughlin, 1990; Hadley, 1994; Jansen & Watter, 2012).”
Rethinking eliminative connectionism. (1998, background): “…are two problems with this measure: first, since the test corpus was ''generated in the same way as the . . . training corpus'' (1991, p. 204), it is not clear to what degree the test corpus actually examined the model's ability to generalize as opposed to its ability to memorize (Hadley, 1994).”
Connectionist sentence processing in perspective (1999, background): “Elman's early work was criticized by Hadley (1994a) for non-systematicity in the selection of test materials with respect to the claimed implicit grammar, so that in some cases the implicit grammar was not even proved weakly equivalent to the one that was initially claimed.”
Learning to Reason with Third-Order Tensor Products (2018, methodology): “They are catastrophically affected by systematic [10, 11] differences between training and test data [12–15].”
Learn more by training less: systematicity in sentence processing by recurrent networks (2006, background): “The extent to which particular connectionist models of language processing show systematicity was investigated by Hadley (1994), who noted that systematic behaviour is basically an issue of learning and generalization: neural networks are trained on a limited number of sentences from which they…”
Darwin's mistake: Explaining the discontinuity between human and nonhuman minds (2008, unknown): “" Many critics have pointed out that Fodor and Pylyshyn's formulation of systematicity was not particularly welldefined or operationally tractable (e.g., Doumas & Hummel 2005; Hadley 1994; Niklasson & van Gelder 1994).”
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation (2020, methodology): “We test for generalization to the targeted grammatical roles not only in simple sentences, but also embedded clauses; this form of generalization is a deﬁning criterion of strong systematicity (Hadley, 1994).”
The Acquisition of Anaphora by Simple Recurrent Networks (2013, background): “(See Marcus 2001 and Hadley 1994 for arguments that humans exploit this kind of ability to universally generalize about individual entities in a variety of domains.)”
Strong systematicity through sensorimotor conceptual grounding: an unsupervised, developmental approach to connectionist sentence processing (2012, background): “Hadley (1994) divided systematicity into what he called weakly and strongly systematic behaviour.”
How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (2021, unknown): “It is often discussed in the context of out-of-distribution generalization (Hadley, 1994; Hupkes et al., 2020; Keysers et al., 2020; Li et al., 2021), typically relying on synthetic datasets to test models’ compositional abilities (Lake and Baroni, 2018; Kim and Linzen, 2020; McCoy et al., 2020).”
How Limited Systematicity Emerges: A Computational Cognitive Neuroscience Approach (Author's Manuscript) (2014, background): “By implication, systematicity also comes in degrees (Hadley, 1994; Niklasson & van 14 How Limited Systematicity Emerges Gelder, 1994).”
The Problem of Rapid Variable Creation (2009, background): “Indeed, an analogous inability arises with the neurally inspired models that I have offered as solutions to the challenge of strong semantic systematicity (cf. Hadley & Cardei, 1999; Hadley & Hayward, 1997; Hadley, 1994).”
A Usage-Based Approach to Recursion in Sentence Processing (2009, background): “Indeed, Hadley (1994) suggested that connectionist models could not afford the kind of generalization abilities necessary to account for human language processing (see Marcus, 1998, for a similar critique).”
Distributed representations for extended syntactic transformation (2000, background): “1992, Hadley 1992, 1994a, b, 1999, Niklasson and Sharkey 1992, Niklasson and van Gelder 1994a, b, Phillips 1994, 1998, 2000, Browne and Pilkington 1994, Phillips and Halford 1997, Bodén and Niklasson, 2000).”