Teaching Machines to Describe Images with Natural Language Feedback

By Huan Ling, Sanja Fidler. 2017

29 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, background): “The QApedia framework (Kratzwald and Feuerriegel, 2019) continuously reﬁnes and updates its content repository using humans in the loop, while human feedback loops have been used to improve image captioning systems (Ling and Fidler, 2017).”
Learning to summarize from human feedback (2020, background): “[17] S.”
Training Classifiers with Natural Language Explanations (2018, background): “Ling and Fidler (2017) use natural language explanations to assist in supervising an image captioning model.”
Training Language Models with Language Feedback (2022, background): “…al., 2022; Ouyang et al., 2022, inter alia ), drive exploration (Tam et al., 2022), infer reward functions (Lin et al., 2022; Sumers et al., 2021; Fidler et al., 2017, inter alia ), and train the model via strong supervision (Andreas et al., 2017; Kaplan et al., 2017), reward shaping (Goyal et…”
Learning Rewards from Linguistic Feedback (2020, methodology): “A somewhat smaller body of work investigates such open-ended language: to correct captioning models (Ling and Fidler 2017), capture environmental characteristics (Narasimhan, Barzilay, and Jaakkola 2018), or improve hindsight replay (Cideron et al. 2019).”
Training Language Models with Language Feedback at Scale (2023, background): “…(“instruction following” , inter alia ) driving exploration (Tam et al., 2022), inferring reward functions (Lin et al., 2022; Sumers et al., 2021; Fidler et al., 2017, inter alia ), and training a model via strong supervision (Andreas et al., 2017; Kaplan et al., 2017), reward shaping (Goyal et…”
Improving Code Generation by Training with Natural Language Feedback (2023, background): “…that have explored supervision via natural language in other ways, such as via explanations (Camburu et al., 2018; Hase & Bansal, 2021; Pruthi et al., 2021; Lampinen et al., 2022, inter alia ) and as part of RL systems (Fidler et al., 2017; Luketina et al., 2019; Lin et al., 2020, inter alia ).”
Syntax-Aware Action Targeting for Video Captioning (2020, background): “Ling and Fidler [30] explored teaching machines to describe images by correcting mistaken phrases.”
Discriminability Objective for Training Descriptive Captions (2018, background): “The ideal way to measure discriminability is to pass it to human and get feedback from them, like in [45].”