Dealing with non-stationary environments using context detection

By Bruno Castro da Silva, Eduardo W. Basso, Ana Lúcia Cetertich Bazzan, Paulo Martins Engel. 2006

201 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “…task states is being able to detect change points or boundaries between significant switches without an oracle as in (Padakandla et al., 2019; Da Silva et al., 2006; Rosman and Ramamoorthy, 2012; Hadoux, Beynier, and Weng, 2014b; Li, Gu, Zhu, and Zhang, 2019; Kessler, Parker-Holder, Ball,…”
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning (2017, background): “…- and AWESOME (Conitzer & Sandholm, 2007) try to tackle the nonstationarity by tracking and conditioning each agents’ learning process on their teammates’ current policy, while Da Silva et al. (2006) propose detecting and tracking different classes of traces on which to condition policy learning.”
Solving Hidden-Semi-Markov-Mode Markov Decision Problems (2014, background): “Planning in such non-stationary environments has already been studied in the MDP framework [8] and in the reinforcement learning framework [7, 9, 15].”
An exploration strategy for non-stationary opponents (2017, background): “Another related approach is the reinforcement learning (RL)with context detection [15].”
Opportunities for multiagent systems and multiagent reinforcement learning in traffic control (2009, background): “[7,10,11]; [72,71] Camponogara and Kraus [18]; Nunes and Oliveira [52]; Steingrover et al.”
Lifelong Hyper-Policy Optimization with Multiple Importance Sampling Regularization (2021, background): “In this second case, many works focus on the detection of abrupt changes in the dynamics (da Silva et al. 2006; Hadoux, Beynier, and Weng 2014) or on scenarios where the non-stationarity arises from switching between stationary dynamics, where the number of such dynamics is known (Choi, Yeung, and…”
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning (2019, methodology): “Before advances in deep RL, approaches like tracking (Sutton et al. , 2007) and context detection (Da Silva et al. , 2006) were proposed to allow quicker adaptation in non-stationary environments.”
Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning (2019, unknown): “…either under limited types of scenarios (e.g. mean-ﬁeld games (Yang et al., 2018)), pre-deﬁned opponent strategies (e.g. Tit-fot-Tat in iterated Prisoner’s Dilemma (Foerster et al., 2018)), or in cases where opponents are assumed to constantly return to the same strategy (Da Silva et al., 2006).”
A review on agent-based technology for traffic and transportation (2013, unknown): “In Silva et al .”
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity (2017, methodology): “In this case, the learning agent starts without predefined opponent strategies or policies (Da Silva et al., 2006; Hernandez-Leal et al., 2014a).”
Deep Reinforcement Learning amidst Continual Structured Non-Stationarity (2021, background): “Algorithms that study the HM-MDP setting aim to quickly detect changes in the environment (Da Silva et al., 2006; Hadoux et al., 2014; Banerjee et al., 2017; Padakandla et al., 2020), while LILAC aims to anticipate future changes and adapt as they happen.”
Learning against sequential opponents in repeated stochastic games (2017, background): “Reinforcement Learning with Context detection (RL-CD) [10] works in single agent tasks with a changing environment, i.e., different contexts .”
A Multitask Representation Using Reusable Local Policy Templates (2012, background): “…(Yu and Mannor 2009), viewing the non-stationarity as arising from mode switches within a set of MDPs (Choi, Yeung, and Zhang 1999) and modifying the value iteration process to accommodate online estimates of transition/reward statistics (Jaulmes, Pineau, and Precup 2005; da Silva et al. 2006).”
Deep Reinforcement Learning amidst Lifelong Non-Stationarity (2020, background): “Previous work both within sequential decision making in changing environments [9, 21, 4] and meta-learning within changing data streams [24] may enable a version of LILAC capable of handling unobserved changepoints, and this setting is likely a fruitful direction for future research.”
Prediction-Based Multi-Agent Reinforcement Learning in Inherently Non-Stationary Environments (2017, methodology): “RL with Context Detection (RL-CD) [Silva et al. 2006] attempts to address environment dynamics with a context detection-based approach that builds on multiple partial models.”
Transient Non-stationarity and Generalisation in Deep Reinforcement Learning (2020, background): “For non-stationary environments during agent deployment, the approach is typically to detect such changes and respond accordingly (Choi et al., 2000; Da Silva et al., 2006; Doya et al., 2002).”
Sequential Decision-Making under Non-stationary Environments via Sequential Change-point Detection (2014, methodology): “In the literature, such kind of non-stationary environments has already been actively investigated [4, 6, 14] in the RL setting.”