Performance bounds for linear stochastic control

By Yang Wang, Stephen P. Boyd. 2009

82 citations recorded by the source corpus.

We develop computational bounds on performance for causal state feedback stochastic control with linear dynamics, arbitrary noise distribution, and arbitrary input constraint set. This can be very useful as a comparison with the performance of suboptimal control policies, which we can evaluate using Monte Carlo simulation. Our method involves solving a semidefinite program (a linear optimization problem with linear matrix inequality constraints), a convex optimization problem which can be efficiently solved. Numerical experiments show that the lower bound obtained by our method is often close to the performance achieved by several widely-used suboptimal control policies, which shows that both are nearly optimal. As a by-product, our performance bound yields approximate value functions that can be used as control Lyapunov functions for suboptimal control policies.

Subfields: Control and Systems Engineering, Numerical Analysis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Numerical Analysis

Citations in the corpus

Variational Bayesian Reinforcement Learning with Regret Bounds (2018, methodology): “This is a dimensionality reduction and generalization strategy that, in the no uncertainty case, has theoretical guarantees [25, 82], and has been used successfully in continuous control [109, 110, 66, 67, 60].”
Min-max approximate dynamic programming (2011, methodology): “To see this we note that infv supα∈A ( ℓ(z, v) + γ EVα(f(z, u, w)) ) ≤ infv ( ℓ(z, v) + γ E supα∈A Vα(f(z, u, w)) ) ≤ infv ( ℓ(z, v) + γ EV ⋆(f(z, u, w)) ) = (T V ⋆)(z) = V ⋆(z), where the first inequality is due to Fatou’s lemma [21], the second inequality follows from the monotonicity of…”
Fast Model Predictive Control Using Online Optimization (2010, methodology): “We can work out the linear control obtained using the associated LQR problem, ignoring the constraints, or using the techniques described in [44], and then project this trajectory a small distance into the feasible set (which is a box).”