One Big Net For Everything

By J. Urgen Schmidhuber. 2018

34 citations recorded by the source corpus.

I apply recent work on "learning to think" (2015) and on PowerPlay (2011) to the incremental training of an increasingly general problem solver, continually learning to solve new tasks without forgetting previous skills. The problem solver is a single recurrent neural network (or similar general purpose computer) called ONE. ONE is unusual in the sense that it is trained in various ways, e.g., by black box optimization / reinforcement learning / artificial evolution as well as supervised / unsupervised learning. For example, ONE may learn through neuroevolution to control a robot through environment-changing actions, and learn through unsupervised gradient descent to predict future inputs and vector-valued reward signals as suggested in 1990. User-given tasks can be defined through extra goal-defining input patterns, also proposed in 1990. Suppose ONE has already learned many skills. Now a copy of ONE can be re-trained to learn a new skill, e.g., through neuroevolution without a teacher. Here it may profit from re-using previously learned subroutines, but it may also forget previous skills. Then ONE is retrained in PowerPlay style (2011) on stored input/output traces of (a) ONE's copy executing the new skill and (b) previous instances of ONE whose skills are still considered worth memorizing. Simultaneously, ONE is retrained on old traces (even those of unsuccessful trials) to become a better predictor, without additional expensive interaction with the enviroment. More and more control and prediction skills are thus collapsed into ONE, like in the chunker-automatizer system of the neural history compressor (1991). This forces ONE to relate partially analogous skills (with shared algorithmic information) to each other, creating common subroutines in form of shared subnetworks of ONE, to greatly speed up subsequent learning of additional, novel but algorithmically related skills.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Large Language Models

Citations in the corpus

Recurrent World Models Facilitate Policy Evolution (2018, background): “A recent One Big Net [84] extension of the C–”
Recurrent Independent Mechanisms (2019, methodology): “This way, training an ensemble of these RIMs can be more robust to out-of-distribution or distractor observations than training one big homogeneous neural network (Schmidhuber, 2018).”
World Models (2018, background): “A recent One Big Net (Schmidhuber, 2018) extension of the C–”
Progress & Compress: A scalable framework for continual learning (2018, background): “Some ideas that could serve as inspiration for future work on the continual learning problem can also be found in Schmidhuber (2018).”
Continual Learning with Adaptive Weights (CLAW) (2019, background): “Data belonging to different tasks could potentially be non i.i.d. (Schlimmer & Fisher, 1986; Sutton & Whitehead, 1993; Ring, 1997; Schmidhuber, 2013; Nguyen et al., 2018; Schmidhuber, 2018).”
A Generalist Agent (2022, methodology): “Recent position papers advocate for highly generalist models, notably Schmidhuber (2018) proposing one big net for everything, and Bommasani et al. (2021) on foundation models.”
On the link between conscious function and general intelligence in humans and machines (2022, methodology): “On the one hand, there is the perspective that models which are orders of magnitude larger, and trained on orders of magnitude more data, will be able to display properties of few-shot generalization like those found in humans (Schmidhuber, 2018).”
Unity: A General Platform for Intelligent Agents (2018, background): “This is often the case for human task acquisition in the real world, and the ability to learn new tasks over time is seen as a key-component of continual learning (Ring, 1994), and ultimately systems capable of artiﬁcial general intelligence (Schmidhuber, 2015; Schmidhuber, 2018).”
An Embodied Generalist Agent in 3D World (2023, unknown): “Building one generalist model that can handle comprehensive tasks like humans has been a long-existing pursuit in artificial intelligence and neuroscience (Lake et al., 2015; 2017; Zhu et al., 2020; Mountcastle, 1979; Schmid-huber, 2018; Huang et al., 2022a).”
Reinforcement Learning Upside Down: Don't Predict Rewards - Just Map Them to Actions (2019, result): “Similar for the “One Big Net” [54] and a recent study of incremental skill learning with feedforward networks [4].”