Design for an Optimal Probe

By Michael O. Duff. 2003

53 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

Learning Parameterized Skills (2012, methodology): “Duff [21] proposes a Bayesian approach for the case in which prior uncertainties about the transition probabilities are available; Abbeel and Ng [1] present a method for computing a near-optimal policy assuming that demonstrations by a teacher are available.”
EPOpt: Learning Robust Neural Network Policies Using Model Ensembles (2016, methodology): “Previous methods have also suggested treating uncertain model parameters as unobserved state variables in a continuous POMDP framework, and solving the POMDP to get optimal exploration-exploitation trade-off (Duff, 2003; Porta et al., 2006).”
Variance-Based Rewards for Approximate Bayesian Reinforcement Learning (2010, background): “A Bayes-optimal agent solves the explore– exploit dilemma by explicitly including information about its belief in its state representation and incorporating information changes into its plans (Duﬀ, 2003).”
Comparing Action-Query Strategies in Semi-Autonomous Agents (2011, background): “Although desirable, nonmyopic optimization would require solving an intractable sequential decision-making problem (see Duff 2003) to find an optimal action-query policy.”
Bayes-Adaptive Simulation-based Search with Value Function Approximation (2014, methodology): “A function approximation scheme in the context of BA planning has been considered by Duff [7], in an offline actor-critic paradigm.”
Learning and Using Models (2012, background): “Duff (2003) presents an ’optimal probe’ that solves the exploration problem optimally, using an augmented statespace that includes both the agent’s state in the world and its beliefs over its models (called a belief state MDP).”
Multi-task reinforcement learning: a hierarchical Bayesian approach (2007, background): “Hierarchical Bayesian RL is also related to Bayesian Reinforcement Learning (Dearden et al., 1998a; Dearden et al., 1998b; Strens, 2000; Du., 2003), where the goal is to give a principled solution to the problem of exploration by explicitly modeling the uncertainty in the rewards, state-transitio”
An intrinsic reward mechanism for efficient exploration (2006, methodology): “Another direction of future research is to find better solutions to our formulation, perhaps by identifying useful features for state approximation on the internal state dimension, as in Duff (2003).”
An analytic solution to discrete Bayesian reinforcement learning (2006, background): “…(Kaelbling, 1993; Meuleau & Bourgine, 1999), Normal-Gamma distributions (Dearden et al., 1998), linear combinations of hyperparameters (Duff, 2003) and sampling (Dearden et al., 1999; Strens, 2000; Wang et al., 2005) have been proposed, but they tend to be computationally intensive at run time,…”