Optimal control of Markov processes with incomplete state information

By K. Åström. 1965

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Shaping Belief States with Generative Environment Models for RL (2019, background): “A formal way of describing agents in partially observed environments is through the notion of partially observable Markov decision processes [4, 5] (POMDPs).”
Heuristic Search Value Iteration for POMDPs (2004, methodology): “To initialize the upper bound, we assume full observability and solve the MDP version of the problem [Astrom, 1965].”
Perseus: Randomized Point-based Value Iteration for POMDPs (2005, background): “Partially observable Markov decision processes (POMDPs) provide a rich mathematical framework for acting optimally in such partially observable and stochastic environments (Dynkin, 1965; ˚Astr¨om, 1965; Aoki, 1965; Sondik, 1971; Lovejoy, 1991; Kaelbling, Littman, & Cassandra, 1998).”
Learning to plan with uncertain topological maps (2020, unknown): “Since the agent does not have access to the actual state of the world, it acts based solely on its entire history of actions and observations, or the corresponding belief state, i.e., the posterior probability distribution over the states given the history [2,50].”
On the undecidability of probabilistic planning and related stochastic optimization problems (2003, background): “Early papers on POMDP problem formulations include those by Drake [16], Astrom [3], and Aoki [2].”
Exact Combinatorial Optimization with Graph Convolutional Neural Networks (2019, background): “…note that this is really only a subset of the solver state, which technically turns the process into a partially-observable Markov decision process [6], but also that excellent variable selection policies such as strong branching are able to do well despite relying only on a subset of the solver…”
Convex Optimization: Algorithms and Complexity (2015, unknown): “The Partially Observable Markov Decision Process (POMDP) is an extension of MDP to the case where the state of the system is not necessarily observable [Astrom, 1965, Smallwood and Sondik, 1973, Kaelbling et al., 1998].”
The free energy principle made simpler but not too simple (2022, unknown): “Lastly, if we remove intrinsic value, we are left with maximising extrinsic value or expected utility, which underwrites expected utility theory [66], game theory, optimal control [143,144] and reinforcement learning [63].”
Acting Optimally in Partially Observable Stochastic Domains (1994, background): “The belief mdp is Markov Astrom 1965, that is, having information about previous belief states cannot improve the choice of action.”
Feedback control of quantum state reduction (2004, background): “It is a standard idea in stochastic control that an output feedback control problem can be converted into a state feedback problem for the filter [9], [10].”
History Compression via Language Models in Reinforcement Learning (2022, background): “Recently, reinforcement learning (RL) for partially observable Markov decision processes (POMDPs; Åström, 1964; Kaelbling et al., 1998) became popular and attracted a lot of attention through mastering complex games such as DotA (Berner et al.”
A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers (2022, background): “Since the belief is a sufficient statistic, optimal policies for the original POMDP may be found by solving an equivalent continuous-space MDP over B(S) [70], [72].”
Anytime Point-Based Approximations for Large POMDPs (2006, background): “A well-known fact is that this history does not need to be represented explicitly, but can instead be summarized via a belief distribution (Ästrom, 1965), which is the following posterior probability distribution: bt(s) := Pr(st = s | zt, at−1, zt−1, . . . , a0, b0).”