Multi-armed bandits with episode context

By Christopher D. Rosin. 2011

322 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Reinforcement Learning

Citations in the corpus

Learning to Search with MCTSnets (2018, methodology): “We also tested a stronger model-based baseline that uses both a pre-learned policy and value network with a PUCT-like rule (Silver et al., 2016), this achieved ≈ 65 % with 25 sims.”
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero (2019, methodology): “It uses a variant of PUCT (Rosin, 2011) to balance exploration (i.e. visiting game states suggested by the prior policy) and exploitation (i.e. visiting game states that have a high value), trading off between the two with a c puct constant.”
Mastering the game of Go with deep neural networks and tree search (2016, methodology): “The strongest current Go programs are based on MCTS 13–15 MCTS has previously been combined with a policy that is used to narrow the beam of the search tree to high-probability moves 13 ; or to bias the bonus term towards high-probability moves 48 .”
Improve Mathematical Reasoning in Language Models by Automated Process Supervision (2024, unknown): “The construction ends either when the search count reaches a predetermined limit or when no additional rollout candidates are available in the pool.”
Goal-Directed Planning via Hindsight Experience Replay (2022, methodology): “Furthermore, AlphaZero replaces the classic UCT algorithm with PUCT (Rosin, 2011) and performs the selection according to: .”
Mastering the game of Go without human knowledge (2017, background): “Each simulation starts from the root state and iteratively selects moves that maximize an upper confidence bound Q ( s , a ) + U ( s , a ), where U ( s , a ) ∝ P ( s , a ) / (1 + N ( s , a )) (refs 12, 24), until a leaf node s ′ is encountered.”
Large Language Models as Commonsense Knowledge for Large-Scale Task Planning (2023, methodology): “[32, 33] created PUCT that applies learned policy prior in PUCB [43] to bias the action selection and avoid exploring each action at least”
Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training (2023, unknown): “…node) is selected according to a t = arg max a ( Q ( s t , a ) + U ( s t , a )) where U ( s t , a ) is calculated by a variant of PUCT algorithm (Rosin, 2011): where N ( s, a ) is the visit count of selecting action a at node s , and c puct = log(( b N ( s, b ) + c base + 1) /c base ) + c init…”