VisualBERT: A Simple and Performant Baseline for Vision and Language

By Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang. 2019

2,355 citations recorded by the source corpus.

We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks. VisualBERT consists of a stack of Transformer layers that implicitly align elements of an input text and regions in an associated input image with self-attention. We further propose two visually-grounded language model objectives for pre-training VisualBERT on image caption data. Experiments on four vision-and-language tasks including VQA, VCR, NLVR2, and Flickr30K show that VisualBERT outperforms or rivals with state-of-the-art models while being significantly simpler. Further analysis demonstrates that VisualBERT can ground elements of language to image regions without any explicit supervision and is even sensitive to syntactic relationships, tracking, for example, associations between verbs and image regions corresponding to their arguments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Mohit Bansal
  3. Percy Liang
  4. Luke S. Zettlemoyer
  5. Yejin Choi
  6. Yoshua Bengio
  7. Zhe Gan
  8. Quoc V. Le

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Cordelia Schmid

In Multimodal Learning

  1. Zhe Gan
  2. Lijuan Wang
  3. Linjie Li
  4. Jianfeng Gao
  5. Mohit Bansal
  6. Devi Parikh
  7. Jingjing Liu
  8. Zicheng Liu

Citations in the corpus