World Discovery Models
By Mohammad Gheshlaghi Azar, Bilal Piot, B. '. Pires, Jean-Bastien Grill, Florent Altch'e, R´emi Munos. 2019
28 citations recorded by the source corpus.
As humans we are driven by a strong desire for seeking novelty in our world. Also upon observing a novel pattern we are capable of refining our understanding of the world based on the new information---humans can discover their world. The outstanding ability of the human mind for discovery has led to many breakthroughs in science, art and technology. Here we investigate the possibility of building an agent capable of discovering its world using the modern AI technology. In particular we introduce NDIGO, Neural Differential Information Gain Optimisation, a self-supervised discovery model that aims at seeking new information to construct a global view of its world from partial and noisy observations. Our experiments on some controlled 2-D navigation tasks show that NDIGO outperforms state-of-the-art information-seeking methods in terms of the quality of the learned representation. The improvement in performance is particularly significant in the presence of white or structured noise where other information-seeking methods follow the noise instead of discovering their world.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Yoshua Bengio
- Nicolas Manfred Otto Heess
- Danijar Hafner
- Jimmy Ba
- R´emi Munos
Citations in the corpus
- Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning (2020, methodology): “…have been used to generate intrinsic motivation signal for exploration (Houthooft et al., 2016; Pathak et al., 2017; Burda et al., 2019; Azar et al., 2019; Puigdomènech Badia et al., 2020) and as the basis for search models in planning agents (Oh et al., 2017; Amos et al., 2018; Hafner…”
- Action and Perception as Divergence Minimization (2020, background): “State representations lead to agents that disambiguate unobserved environment states, for example by opening doors to see objects behind them, such as in active inference (Da Costa et al., 2020), INDIGO (Azar et al., 2019), and DVBF-LM (Mirchev et al., 2018).”
- Meta-learning curiosity algorithms (2020, background): “, 2007; Schmidhuber, 2008; Azar et al., 2019), generating diverse skills (Eysenbach et al., 2018), stochastic neural networks (Florensa et al., 2017; Fortunato et al., 2017), count-based exploration (Tang et al., 2017) or object-based curiosity measures (Forestier & Oudeyer, 2016). Finally, part of ”