Novelty or Surprise?

By Andrew Barto, Marco Mirolli, Gianluca Baldassarre. Frontiers in Psychology, 2013

369 citations recorded by the source corpus.

Novelty and surprise play significant roles in animal behavior and in attempts to understand the neural mechanisms underlying it. They also play important roles in technology, where detecting observations that are novel or surprising is central to many applications, such as medical diagnosis, text processing, surveillance, and security. Theories of motivation, particularly of intrinsic motivation, place novelty and surprise among the primary factors that arouse interest, motivate exploratory or avoidance behavior, and drive learning. In many of these studies, novelty and surprise are not distinguished from one another: the words are used more-or-less interchangeably. However, while undeniably closely related, novelty and surprise are very different. The purpose of this article is first to highlight the differences between novelty and surprise and to discuss how they are related by presenting an extensive review of mathematical and computational proposals related to them, and then to explore the implications of this for understanding behavioral and neuroscience data. We argue that opportunities for improved understanding of behavior and its neural basis are likely being missed by failing to distinguish between novelty and surprise.

Subfields: Cell Biology, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/1695ff30aa141088a019fc29df6d128d6add06f5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cell Biology

In Cognitive Neuroscience

Citations in the corpus

Latent Space Autoregression for Novelty Detection (2018, background): “According to cognitive psychology [4], novelty can be expressed either in terms of capabilities to remember an event or as a degree of surprisal [39] aroused by its observation.”
Active Inference: A Process Theory (2017, background): “…regarded as the drive for novelty-seeking behavior (Wittmann, Daw, Seymour, & Dolan, 2008; Krebs, Schott, Sch¨utze, & D¨uzel, 2009; Schwarten-beck et al., 2013), in which we anticipate uncertainty that can be resolved (e.g., opening a birthday present: see also Barto, Mirolli, & Baldassarre, 2013).”
GRAIL: A Goal-Discovering Robotic Architecture for Intrinsically-Motivated Learning (2016, background): “Note that we are using the IM signal for motivation (i.e., as a drive to select and execute a goal/behavior) rather than for learning (i.e., to update connection weights) as in most models using IMs.”
AI Autonomy: Self-initiated Open-world Continual Learning and Adaptation (2022, background): “Novelty-based IMs try to detect novel items and direct attention to novel items in exploration.”
Continuous Control with Action Quantization from Demonstrations (2021, background): “non-Markovian, optimizing for something different than a reward function like curiosity (Barto et al., 2013), or they can be in the process of learning how to interact with the environment.”
Intrinsic motivations and open-ended development in animals, humans, and robots: an overview (2014, background): “The theoretical contribution of Barto et al. (2013) argues for the importance of distinguishing between novelty and surprise on the basis of a comprehensive analysis of the computational literature related to the two.”
The free energy principle made simpler but not too simple (2022, neutral): “7, such as intrinsic motivation, artificial curiosity [136–138] and optimal control [93,95,131].”
Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. (2016, background): “, 2013; Kidd and Hayden, 2015), states of curiosity are often associated to a psychological interest for activities or stimuli that are surprising, novel, of intermediate complexity, or characterized by a knowledge gap or by errors in prediction, which are features that can themselves be quantified ”
Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning (2017, background): “The Bayesian surprise associated with a transition is the reduction in uncertainty over possibly dynamics models from observing it ([1],[8]): DKL (P (φ|ht, at, st+1)||P (φ|ht)) .”