Active Exploration in Dynamic Environments

By Sebastian Thrun, K. Möller. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Active Learning with Statistical Models (1996, methodology): “There are many heuristics for choosing ~ x based on intuition, including choosing places where we don't have data [Whitehead, 1991], where we perform poorly [Linden andWeber, 1993], where we have low con dence [Thrun and Moller, 1992], where we expect it to change our model [Cohn et al, 1990], and ”
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016, neutral): “while the name was coined in [190]. However, active SLAM has its roots in ideas from artiﬁcial intelligence and robotic exploration that can be traced back to the early eighties (c.f. [19]). Thrun in [300] and [297] concluded that solving the exploration-exploitation dilemma, i.e., ﬁnding a balance ”
Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time (1993, methodology): “The same philosophy is also used by Thrun and Moller (1992) .”
Learning to Act Using Real-Time Dynamic Programming (1995, background): “Kaelbling [34], Lin [47], Moore [55], Schmidhuber [63], Sutton [69], Thrun [79], and Thrun and Moller [78] discuss these and other possibilities.”
Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges (2010, background): “…evaluate the utility of given sampling candidates, such as the maximization of prediction errors [22], the local density of already queried points [40], the maximization of the decrease of global model variance [2], expected improvement [11], or maximal uncertainty of the model [41], among others.”
Kernel-Based Reinforcement Learning (2004, methodology): “Reinforcement learning has been applied successfully to a variety of practical applications including prominent examples such as Tesauro’s Neurogammon or Singh and Bertsekas’ dynamic channel allocation algorithms ( Tesauro, 1989; Singh & Bertsekas, 1997).”
A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers (2022, background): “Thrun and Möller [25] demonstrate that in order to solve robotic exploration, agents have to switch between two opposite principles depending on the expected costs and gains: exploring new areas and revisiting those already seen, i.”
Understanding the origin of information-seeking exploration in probabilistic objectives for control (2021, background): “…changing reward structures where the exploration-exploitation dilemma can never be fully resolved, since the environment, and thus the estimates of value, may always be changing and thus there is always a need to explore (Cohen et al., 2007; Sun, Gomez, & Schmidhuber, 2011; Thrun & Möller, 1991).”
Reinforcement Learning: an Overview (2000, background): “It is necessary to find a compromise therefore and an abundant literature is dedicated to this problem [9, 19, 29, 36].”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “Directed exploration strategies — counter-based (Sato et al., 1990), error-and-counter based (Thrun and Moller, 1992), and recency-based (Sutton, 1990) — can overcome this drawback.”
Continual learning in reinforcement environments (1995, background): “Even more di cult (and more realistic) are environments with continuous-valued, distributed sense and action vectors [3, 5, 72, 92, 107].”
Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints (2013, background): “, 1996), or maximal uncertainty of the model (Thrun and Moller, 1992), among others.”
An information-theoretic approach to curiosity-driven reinforcement learning (2012, background): “[25,13]) A different strategy, which yields interesting theoretical results, is that of optimism in the face of uncertainty: if a state has not been visited sufficiently for the agent to be familiar with it, it is automatically considered good, so the agent will be driven towards it.”