Preference learning with Gaussian processes

By Wei Chu, Zoubin Ghahramani. 2005

481 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Learning Dynamic Robot-to-Human Object Handover from Human Feedback (2016, methodology): “It was shown in [7] that minimizing J w.r.t. ˆ R is a convex problem in case there is only preference based feedback ( M = 0).”
Research Priorities for Robust and Beneficial Artificial Intelligence (2015, background): “Other approaches could use diﬀerent assumptions about underlying cognitive models of the actor whose preferences are being learned [75], or could be explicitly inspired by the way humans acquire ethical values.”
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010, methodology): “1 are based on [Chu and Ghahramani, 2005b], which presents a preference learning method using probit models and Gaussian processes.”
Active Reward Learning (2014, background): “Chu and Ghahramani [7] introduced the use of GPs to preference rankings.”
Bayesian Active Learning for Classification and Preference Learning (2011, methodology): “Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”
Interactive machine learning for health informatics: when do we need the human-in-the-loop? (2016, background): “The valuation functions for r and c can be modeled as follows: In these functions the noise terms are Gaussian, hence, a non-parametric Gaussian process prior to the unknown mean valuation can be assigned [64].”
A Bayesian Approach for Policy Learning from Trajectory Preference Queries (2012, methodology): “To translate the evaluation into an expert response we borrow from previous work [6].”
Collaborative Gaussian Processes for Preference Learning (2012, methodology): “Bayesian methods can be used to learn f , for example, by modeling f independently for each user as a draw from a Gaussian process (GP) prior [4].”
Coactive Learning (2015, background): “More closely related is learning with pairs of examples (Herbrich, Graepel, & Obermayer, 2000; Freund, Iyer, Schapire, & Singer, 2003; Chu & Ghahramani, 2005), since it circumvents the need for absolute ranks; only relative orderings are required.”
Programming by Feedback (2014, background): “Gaussian perturbation (Chu & Ghahramani, 2005; Wilson et al., 2012) or following the Luce-Shepard model (Luce, 1959; Shepard, 1957).”
Preference-based interactive multi-document summarisation (2019, background): “Simpson and Gurevych (2018) develop an improved Gaussian process preference learning (Chu and Ghahramani 2005) algorithm to learn an argument con-vincingness ranker from noisy preferences.”
Building Naturalistic Emotionally Balanced Speech Corpus by Retrieving Emotional Speech from Existing Podcast Recordings (2019, background): “The study of Chu and Ghahramani [60] provides further details on this”
Online Structured Prediction via Coactive Learning (2012, background): “Closely related is learning with pairs of examples (Herbrich et al., 2000; Freund et al., 2003; Chu & Ghahramani, 2005) where absolute ranks are not needed; however, existing approaches require an iid assumption and typically perform batch learning.”
APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning (2018, background): “improved Gaussian process preference learning (Chu and Ghahramani, 2005) for learning to rank arguments in terms of convincingness from crowdsourced annotations.”
Semi-Supervised Multi-Task Regression (2009, background): “The model in [32] is for classification and the constraints there refer to label preference.”
Learning to rank for information retrieval (2009, unknown): “Example algorithms belonging to the pointwise approach include [24, 25, 26, 31, 33, 34, 49, 53, 73, 78, 90, 114].”
Collaborative hyperparameter tuning (2013, methodology): “The method of choice for Algorithm B in Figure 4 is the Gaussian process-based ranking algorithm of Chu & Ghahramani (2005) since it provides a way of simultaneously estimating the values of a hidden f and tuning the GP hyperparameters in a double optimization loop.”
Gaussian Process Preference Elicitation (2010, methodology): “Their approach is based on the model in [13].”
Dueling Posterior Sampling for Preference-Based Reinforcement Learning (2019, methodology): “Finally, we show how to extend the preference-based Gaussian process model defined in Chu and Ghahramani (2005) from the dueling bandit setting to the PBRL setting to perform credit assignment.”