The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning

By L. Deng. 2006

1,283 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

In Mathematical Optimization

Citations in the corpus

Self-Supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation (2017, methodology): “We also evaluated action selection using the cross entropy method [30], but empirically found no difference in performance.”
Deep visual foresight for planning robot motion (2016, methodology): “This corresponds to the CEM stochastic optimization algorithm.”
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection (2016, methodology): “CEM is a simple derivative-free optimization algorithm that samples a batch of N values at each iteration, fits a Gaussian distribution to M N of these samples, and then samples a new batch ofN from this Gaussian.”
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL (2020, methodology): “QT-OPT (Kalashnikov et al., 2018b) makes use of two iterations of the Cross-Entropy Method (Rubinstein & Kroese, 2013), while CAQL (Ryu et al., 2019) uses Mixed-Integer Programming to ﬁnd the exact maximizing action while also introducing faster approximate alternatives.”
Self-Supervised Visual Planning with Temporal Skip Connections (2017, methodology): “We instead use a stochastic, sampling-based planning method [23, 2], which we extend to handle a mixture of continuous and discrete actions.”
Importance Sampling Policy Evaluation with an Estimated Behavior Policy (2018, methodology): “We obtain a basic policy by optimizing the linear weights of this policy for 10 iterations of the Cross-Entropy method (Rubinstein & Kroese, 2013).”
Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills (2021, positive): “The initial variances of the (x, y, z) end-effector positions and the azimuth angle during the CEM-optimization (Rubinstein & Kroese, 2004) are set to (3cm, 3cm, 6cm, 0.16 rad), respectively.”
Learning to Adapt: Meta-Learning for Model-Based Control (2018, methodology): “However, a variety of alternatives, such as iterative gradient-free algorithms (Blossom, 2006) and gradientbased methods (Li & Todorov, 2004) can also be used to perform this action selection.”
Simulation optimization: a review of algorithms and applications (2014, methodology): “However, the intuition behind the selection of subsequent samples can be shown to be analogous to minimizing the KullbackLeibler divergence (KL-divergence) between the optimal importance sampling distribution and the distribution used in the current iterate [171].”
Reinforcement Learning in Continuous State and Action Spaces (2012, methodology): “…algorithms (Holland, 1962; Rechenberg, 1971; Holland, 1975; Schwefel, 1977; Davis, 1991; Bäck and Schwefel, 1993), simulated annealing (Kirkpatrick, 1984), particle swarm optimization (Kennedy and Eberhart, 1995) and cross-entropy optimization (Rubinstein, 1999; Rubinstein and Kroese, 2004).”
KeyIn: Discovering Subgoal Structure with Keyframe-based Video Prediction (2019, methodology): “After subgoals are selected, we use a CEM based planner to produce rollout trajectories.”
Broadly-Exploring, Local-Policy Trees for Long-Horizon Task Planning (2020, methodology): “We compare to a model-based planner (cross entropy method (CEM) [41]) and a Play-LMP with given goal state.”
Policy Search in a Space of Simple Closed-form Formulas: Towards Interpretability of Reinforcement Learning (2012, methodology): “The work proposed in this paper is also related to approaches aiming to derive optimization schemes for screening policy spaces, such as gradient-free techniques using cross-entropy optimization [25, 20], genetic algorithms [12] and more specifically related to our work, genetic programming algorith”
Improved algorithms for rare event simulation with heavy tails (2006, background): “2)CE means a Pareto f* with a* = 1/log u, as suggested by a cross-entropy argument similar to that of [9] and [19] (we omit the details of the derivation), and (1.”