A Separation Principle for Control in the Age of Deep Learning

By Alessandro Achille, Stefano Soatto. 2017

33 citations recorded by the source corpus.

We review the problem of defining and inferring a state for a control system based on complex, high-dimensional, highly uncertain measurement streams, such as videos. Such a state, or representation, should contain all and only the information needed for control and discount nuisance variability in the data. It should also have finite complexity, ideally modulated depending on available resources. This representation is what we want to store in memory in lieu of the data, as it separates the control task from the measurement process. For the trivial case with no dynamics, a representation can be inferred by minimizing the information bottleneck Lagrangian in a function class realized by deep neural networks. The resulting representation has much higher dimension than the data (already in the millions) but is smaller in the sense of information content, retaining only what is needed for the task. This process also yields representations that are invariant to nuisance factors and have maximally independent components. We extend these ideas to the dynamic case, where the representation is the posterior density of the task variable given the measurements up to the current time, which is in general much simpler than the prediction density maintained by the classical Bayesian filter. Again, this can be finitely parameterized using a deep neural network, and some applications are already beginning to emerge. No explicit assumption of Markovianity is needed; instead, complexity trades off approximation of an optimal representation, including the degree of Markovianity.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Provable Representation Learning for Imitation with Contrastive Fourier Features (2021, background): “While some works have proposed learning action representations [8, 30], our work focuses on state representation learning, which is more common in the literature, and whose aim is generally to distill aspects of the observation relevant to control from those relevant only to measurement [3]; see [27”
State Representation Learning for Control: An Overview (2018, methodology): “Following [Achille and Soatto, 2017], a good representation must be suﬃcient, as eﬃcient as possible (i.e., easy to work with, e.g., factorizing the data-generating factors), and minimal (from all possible representations, take the most eﬃcient one).”
Explainability in Deep Reinforcement Learning (2020, background): “Only this way we will produce actionable explanations and more comprehensive frameworks for explainable, trustable and responsible RL that can be deployed in practice.”
Learning Interactive Real-World Simulators (2023, background): “Most systems choose to learn one dynamics model per system in the lower dimensional state space as opposed to in the pixel space [54, 55, 56, 57], which, despite being a simpler modeling problem, limits knowledge sharing across systems.”