Coactive Learning

By Pannagadatta K. Shivaswamy, Thorsten Joachims. 2015

72 citations recorded by the source corpus.

We propose Coactive Learning as a model of interaction between a learning system and a human user, where both have the common goal of providing results of maximum utility to the user. Interactions in the Coactive Learning model take the following form: at each step, the system (e.g. search engine) receives a context (e.g. query) and predicts an object (e.g. ranking); the user responds by correcting the system if necessary, providing a slightly improved - but not necessarily optimal - object as feedback. We argue that such preference feedback can be inferred in large quantity from observable user behavior (e.g., clicks in web search), unlike the optimal feedback required in the expert model or the cardinal valuations required for bandit learning. Despite the relaxed requirements for the feedback, we show that it is possible to adapt many existing online learning algorithms to the coactive framework. In particular, we provide algorithms that achieve O(1/√T) average regret in terms of cardinal utility, even though the learning algorithm never observes cardinal utility values directly. We also provide an algorithm with O(log(T)/T) average regret in the case of λ-strongly convex loss functions. An extensive empirical study demonstrates the applicability of our model and algorithms on a movie recommendation task, as well as ranking for web search.

Subfields: Artificial Intelligence, Computer Networks and Communications, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Management Science and Operations Research

Citations in the corpus

Dueling Posterior Sampling for Preference-Based Reinforcement Learning (2019, background): “n accurate model rather than maximizing utility of decision-making; learning with more structured preference feedback (Radlinski and Joachims, 2005; Shivaswamy and Joachims, 2012; Raman et al., 2013; Shivaswamy and Joachims, 2015), where the learner receives more than one bit of information per pref”
Explanatory Interactive Machine Learning (2019, background): “Akin to coactive learning [33], query instances are accompanied by the the model’s corresponding predictions.”