Linear Bellman combination for control of character animation

By Marco Jorge Tome da Silva, Frédo Durand, Jovan Popovic. 2009

77 citations recorded by the source corpus.

Controllers are necessary for physically-based synthesis of character animation. However, creating controllers requires either manual tuning or expensive computer optimization. We introduce linear Bellman combination as a method for reusing existing controllers. Given a set of controllers for related tasks, this combination creates a controller that performs a new task. It naturally weights the contribution of each component controller by its relevance to the current state and goal of the system. We demonstrate that linear Bellman combination outperforms naive combination often succeeding where naive combination fails. Furthermore, this combination is provably optimal for a new task if the component controllers are also optimal for related tasks. We demonstrate the applicability of linear Bellman combination to interactive character control of stepping motions and acrobatic maneuvers.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Reinforcement Learning with Deep Energy-Based Policies (2017, background): “In the context of linearly solvable MDPs, several prior works have shown that policies trained for different tasks can be composed to create new optimal policies (Da Silva et al., 2009; Todorov, 2009).”
Iterative Training of Dynamic Skills Inspired by Human Coaching Techniques (2014, background): “A control rig simultaneously controls a set of low-level control variables in a coordinated fashion.”
Discovery of complex behaviors through contact-invariant optimization (2012, background): “Others proposed to in-telligently combine individual controllers [Faloutsos et al. 2001; da Silva et al. 2009; Coros et al. 2009; Muico et al. 2011], although they still rely on existence of base controller libraries.”
The Option Keyboard: Combining Skills in Reinforcement Learning (2021, background): “[6] have shown how, in this scenario, one can compute optimal skills corresponding to linear combinations of other optimal skills—a property later explored by Saxe et al.”
Compositionality of optimal control laws (2009, background): “Finally, another group [6] has developed similar ideas independently and in parallel.”