Learning task-state representations
By Yael Niv. 2019
310 citations recorded by the source corpus.
Arguably, the most difficult part of learning is deciding what to learn about. Should I associate the positive outcome of safely completing a street-crossing with the situation ‘the car approaching the crosswalk was red’ or with ‘the approaching car was slowing down’? In this Perspective, we summarize our recent research into the computational and neural underpinnings of ‘representation learning’—how humans (and other animals) construct task representations that allow efficient learning and decision-making. We first discuss the problem of learning what to ignore when confronted with too much information, so that experience can properly generalize across situations. We then turn to the problem of augmenting perceptual information with inferred latent causes that embody unobservable task-relevant information, such as contextual knowledge. Finally, we discuss recent findings regarding the neural substrates of task representations that suggest the orbitofrontal cortex represents ‘task states’, deploying them for decision-making and learning elsewhere in the brain. When crossing the street, you can ignore the color of oncoming cars, but for hailing a taxi color is important. How do we learn what to represent neurally for each task? Here, Niv summarizes a decade of work on representation learning in the brain.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Matthew Riemer
- Irina Rish
- Khimya Khetarpal
- Doina Precup
- A. G. Collins
- Sergey Levine
- Joelle Pineau
- Eric Eaton
Citations in the corpus
- Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Interesting work by Niv (2019) also suggests that the orbitofrontal cortex may be used to represent task-states that are deployed for decision making and learning elsewhere in the brain.”
- Neural Coding of Cognitive Control: The Representational Similarity Analysis Approach. (2020, unknown): “, planning and action preparation) [79,80].”
- The Role of Executive Function in Shaping Reinforcement Learning. (2020, background): “This is accomplished by attention differentially weighing environmental features, assigning a higher weight to task-relevant ones [33] (Figure 2).”