Teaching Machines to Describe Images via Natural Language Feedback

By Huan Ling, Sanja Fidler. 2017

46 citations recorded by the source corpus.

Robots will eventually be part of every household. It is thus critical to enable algorithms to learn from and be guided by non-expert users. In this paper, we bring a human in the loop, and enable a human teacher to give feedback to a learning agent in the form of natural language. We argue that a descriptive sentence can provide a much stronger learning signal than a numeric reward in that it can easily point to where the mistakes are and how to correct them. We focus on the problem of image captioning in which the quality of the output can easily be judged by non-experts. We propose a hierarchical phrase-based captioning model trained with policy gradients, and design a feedback network that provides reward to the learner by conditioning on the human-provided feedback. We show that by exploiting descriptive feedback our model learns to perform better than when given independently written human captions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, neutral): “The QApedia framework (Kratzwald and Feuerriegel, 2019) continuously reﬁnes and updates its content repository using humans in the loop, while human feedback loops have been used to improve image captioning systems (Ling and Fidler, 2017).”
Training Classifiers with Natural Language Explanations (2018, background): “Ling and Fidler (2017) use natural language explanations to assist in supervising an image captioning model.”
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables (2020, methodology): “] exploited NL explanation as additional features for prediction. For example, Srivastava et al. [9] converted NL explanationsinto classiﬁer features to train text classiﬁcation models. Fidler et al. [12] used natural language explanations to assist in supervising an image captioning model. Very rec”
Why Build an Assistant in Minecraft? (2019, methodology): “Similar techniques have been applied for teaching machines to describe images via natural language feedback as well [56].”
Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback (2020, background): “…semantic parsing, learning through interaction (Li et al., 2017a; Hancock et al., 2019; Li et al., 2017b, inter alia) and learning from natural language supervision (Srivastava et al., 2017; CoReyes et al., 2019; Srivastava et al., 2018; Hancock et al., 2018; Ling and Fidler, 2017, inter alia).”
Syntax-Aware Action Targeting for Video Captioning (2020, background): “Ling and Fidler [30] explored teaching machines to describe images by correcting mistaken phrases.”
Interactive Learning from Activity Description (2021, background): “Common approaches include: reduction to reinforcement learning (Goldwasser & Roth, 2014; MacGlashan et al., 2015; Ling & Fidler, 2017; Goyal et al., 2019; Fu et al., 2019; Sumers et al., 2020), learning to ground language to actions (Chen & Mooney, 2011; Misra et al., 2014; Bisk et al., 2016; Liu…”
Discriminability Objective for Training Descriptive Captions (2018, methodology): “e can be viewed as a hard negative mining version of triplet loss [46]. 3.2. Discriminability loss The ideal way to measure discriminability is to pass it to human and get feedback from them, like in [47]. However it is rather costly and very slow to collect. Here, we propose instead to use a pre-tr”
Learning to Caption Images Through a Lifetime by Asking Questions (2018, background): “Related to us is [15], where a teacher observes the captioning agent in a continual setting, and gives natural language feedback when errors occur.”