Review Networks for Caption Generation

By Zhilin Yang, Ye Yuan, Yuexin Wu, William W. Cohen, Ruslan Salakhutdinov. 2016

268 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning (2018, methodology): “A large number of DNN models for image caption generation have been trained and evaluated using COCO captions (Vinyals et al., 2015a; Fang et al., 2015; Xu et al., 2015; Ranzato et al., 2015; Yang et al., 2016; Liu et al., 2017; Ding and Soricut, 2017).”
Semi-Supervised QA with Generative Domain-Adaptive Nets (2017, methodology): “Review networks (Yang et al., 2016b) employ a discriminative model as a regularizer for training a generative model.”
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning (2021, background): “Many improvements to the encoder [11,38,49,75,76,80,81], the decoder [72, 73, 78], and the attention mechanism [8, 13, 24, 33, 36] has since been proposed.”
Attention on Attention for Image Captioning (2019, neutral): “[42, 27, 47, 2, 7, 16, 15] and achieved impressive results.”
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering (2017, methodology): “Typically, these models can be characterized as top-down approaches, with context provided by a representation of a partially-completed caption in the case of image captioning [33, 27, 47, 45], or a representation of the question in the case of VQA [11, 28, 44, 46, 49].”
nocaps: novel object captioning at scale (2019, background): “Recent progress in image captioning, the task of generating natural language descriptions of visual content [9,10,16, 17,41,44], can be largely attributed to the publicly available large-scale datasets of image-caption pairs [5,14,48] as well as steady modeling improvements [4, 24, 35, 46].”
RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning (2020, background): “ReviewerNet [45] also selects in advance which part of the image will be attended, so that the decoder is aware of it from the beginning.”
Improved Image Captioning via Policy Gradient optimization of SPIDEr (2016, background): “, the “Show, Attend and Tell” model of [26], and the “Review Network” model of [27]), leverages attention, which lets the decoder focus on specific parts of the input image when generating a word.”
Semantic Compositional Networks for Visual Captioning (2016, methodology): “This work was followed by [53] which introduced a review module to improve the attention mechanism and [28] which proposed a method to improve the correctness of visual attention.”
Boosting Image Captioning with Attributes (2016, neutral): “9 reviewnet [35] 72.”
Words or Characters? Fine-grained Gating for Reading Comprehension (2016, background): “The gating mechanism can also be viewed as a form of attention mechanism (Bahdanau et al., 2015; Yang et al., 2016b) over two inputs.”
Towards Diverse and Natural Image Descriptions via a Conditional GAN (2017, background): “Along with the recent surge of deep learning technologies, there has been remarkable progress in image captioning over the past few years [11, 29–32].”
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning (2017, background): “…and Zitnick, 2015; De-vlin et al., 2015; Donahue et al., 2015a; Karpa-thy and Li, 2015; Mao et al., 2015; Vinyals et al., 2015; Xu et al., 2015; Yang et al., 2016; Liu et al., 2017a,b), which are mainly differed by the underlying CNN and RNN architectures, and whether or not the attention…”
Image Captioning: Transforming Objects into Words (2019, background): “A family of attention based approaches [26, 30, 28] to image captioning have also been proposed that seek to ground the words in the predicted caption to regions in the image.”
Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks (2017, background): “Yang et al. (2016) proposed a review network to the image captioning, which reviews all the information encoded by the encoder and produces a compact thought vector.”
From Show to Tell: A Survey on Deep Learning-Based Image Captioning (2021, methodology): “[50] supplemented the encoder-decoder framework with a recurrent”
Discriminability Objective for Training Descriptive Captions (2018, background): “Within this generic framework, many efforts [7, 10, 11, 8, 12, 13, 14, 15, 16] explored different encoder-decoder structures, including attention-based models.”
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning (2017, background): “Their work was later followed by (Yang et al. 2016a) and (Liu et al.”
Know More Say Less: Image Captioning Based on Scene Graphs (2019, background): “Yang et al. [65] add a reviewer module that improves the representation passed to the decoder.”