Skip Context Tree Switching

By Marc G. Bellemare, Joel Veness, Erik Talvitie. 2014

45 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Action-Conditional Video Prediction using Deep Networks in Atari Games (2015, background): “One line of work [3, 4] divides game images into patches and applies a Bayesian framework to predict patch-based observations.”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, methodology): “We use a simplified, pixel-level version of the CTS model for Atari 2600 frames proposed by Bellemare et al. (2014), ignoring temporal dependencies.”
Count-Based Exploration with Neural Density Models (2017, methodology): “The CTS density model (Bellemare et al., 2014) is based on the namesake algorithm, Context Tree Switching (Ve-ness et al., 2012), a Bayesian variable-order Markov model.”
Compress and Control (2014, methodology): “Although the emulator generates frames at 60Hz, in our experiments we consider time steps that last 4 consecutive frames, following the existing literature (Bellemare, Veness, and Talvitie 2014; Mnih et al. 2013).”
The Effect of Planning Shape on Dyna-style Planning in High-dimensional State Spaces (2018, methodology): “Despite the introduction of increasingly effective approaches for learning predictive models in Atari Games (Bellemare, Veness, & Bowling, 2013; Bellemare, Veness, & Talvitie, 2014; Oh et al., 2015), to our knowledge this is the ﬁrst time that a learned dynamics model has been successfully used for…”
Deep Variational Reinforcement Learning for POMDPs (2018, methodology): “The exponential growth in t of H t can be addressed, e.g., with sufﬁx trees (McCallum & Ballard, 1996; Shani et al., 2005; Bellemare et al., 2014; Bellemare, 2015; Messias & Whiteson, 2017).”
Self-Correcting Models for Model-Based Reinforcement Learning (2016, unknown): “Even as model-learning methods demonstrate increasing prediction accuracy in high-dimensional domains (e.g. Bellemare et al. 2014, Oh et al. 2015) this rarely corresponds to improvements in control performance.”
Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning (2014, background): “In principle, one could learn a state representation and infer an associated MDP model using frame-observation and action trajectories, but these games are so complex that this is rarely done (though see [4] for a recent attempt).”
Deep Curiosity Search: Intra-Life Exploration Improves Performance on Challenging Deep Reinforcement Learning Problems (2018, background): “If one did not have access to RAM states, position could be inferred via object detection [33] or by implicit state-space representations such as Skip Context Tree Switching [34], the latter of which was shown to work well in a roughly similar counting-based model [9].”
On Bonus Based Exploration Methods In The Arcade Learning Environment (2020, methodology): “We ran a new hyperparameter sweep on CTS and RND using a new training set.”
EMI: Exploration with Mutual Information (2018, methodology): “Bellemare et al. (2016); Ostrovski et al. (2017) utilize density estimation via CTS (Bellemare et al., 2014) generative model and PixelCNN (van den Oord et al., 2016) and derive pseudo-counts as the intrinsic motivation.”