A POMDP formulation of preference elicitation problems

By Craig Boutilier. AAAI/IAAI, 2002

324 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/8b66050af65389875818df7a4dfacfa62e8be7f9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning with Limited Reinforcement : Using Bayes Risk for Active Learning in POMDPs Finale (2012, background): “The preference el icitation literature [44, 45] focuses on learning preferences based on queries, but th ere, learning the rewards is itself the objective: the agent does not have to bala nce between learning a model and performing a task.”
Bayesian Multitask Inverse Reinforcement Learning (2011, background): “A number of inverse reinforcement learning [1, 5, 7, 16, 18, 20, 23] and preference elicitation [4, 6] approaches have been proposed, while multitask learning itself is a well-known problem, for which hierarchical Bayesian approaches are quite natural [13].”
An artificial intelligence perspective on autonomic computing policies (2004, background): “It is well established that humans have difficulty specifying utility functions, and much work has been done to understand and ameliorate the difficulty [ 31 ], [32], [33].”
Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making (2018, methodology): “We also aim to study acquisition functions that pick more than one next item, based on look-ahead information gain [6].”
Potential-based reward shaping for finite horizon online POMDP planning (2016, background): “, active sensing [4,23]), one of the primary goals of the agent is to discover the environment’s hidden state before it acts on its beliefs to achieve tasks and goals.”
Consequences of Misaligned AI (2021, background): “This includes systems that learn from direct comparisons between options (9; 11; 13), demonstrations of optimal behavior (26; 1; 36), corrective feedback (7; 15; 2), and proxy metrics (18).”
Artificial Intelligence and Collective Intelligence (2014, background): “Most people find it hard to articulate their precise utility function, and this has led to techniques for preference elicitation, which usually try to induce a general utility function from a small set of concrete judgements that are easier for people to answer (Boutilier 2002; Gajos and Weld 2005).”
Autonomous agents modelling other agents: A comprehensive survey and open problems (2017, background): “Closely related areas are active learning [242], preference elicitation [45,69], and Bayesian experimental design [73].”
Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs (2008, unknown): “The preference el icitation literature [44, 45] focuses on learning preferences based on queries, but th ere, learning the rewards is itself the objective: the agent does not have to bala nce between learning a model and performing a task.”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, methodology): “For example, Boutilier [9] propose a partially observedMarkov decision process (POMDP) framework as the sequential query strategy.”
Human-Guided Learning for Probabilistic Logic Models (2018, background): “Preferential advice , inspired from preference elicitation approaches (Boutilier, 2002) that can improve learning in domains with systematic noise (where certain types of examples are consistently mislabeled possibly due to human or sensor error).”
Multiobjective Reinforcement Learning: A Comprehensive Overview (2015, background): “This problem is usually called preference elicitation (PE), and its major related work is inverse RL [59], [60].”
Point-Based Value Iteration for Continuous POMDPs (2006, background): “, 2003b), preference elicitation (Boutilier, 2002), spoken-dialog systems (Roy et al.”
A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation (2019, methodology): “Our policy adaptation scheme is related to prior work in preference elicitation [25, 26, 27] or inverse reinforcement learning [28, 29].”
Gaussian Process Preference Elicitation (2010, methodology): “In particular, we have subsampled 50 training users and selected about 5 training pairwise preferences drawn from each of the N = 10 available items.”