Visual Semantic Role Labeling

By Saurabh Gupta, Jitendra Malik. 2015

488 citations recorded by the source corpus.

In this paper we introduce the problem of Visual Semantic Role Labeling: given an image we want to detect people doing actions and localize the objects of interaction. Classical approaches to action recognition either study the task of action classification at the image or video clip level or at best produce a bounding box around the person doing the action. We believe such an output is inadequate and a complete understanding can only come when we are able to associate objects in the scene to the different semantic roles of the action. To enable progress towards this goal, we annotate a dataset of 16K people instances in 10K images with actions they are doing and associate objects in the scene with different semantic roles for each action. Finally, we provide a set of baseline algorithms for this task and analyze error modes providing directions for future work.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Cordelia Schmid
  3. Trevor Darrell
  4. Abhinav Kumar Gupta
  5. Jitendra Malik
  6. Quoc V. Le
  7. Yoshua Bengio
  8. Jianfeng Gao

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Jitendra Malik
  3. Trevor Darrell
  4. Dacheng Tao
  5. Ming-Hsuan Yang
  6. Cordelia Schmid
  7. Xiaogang Wang
  8. Andrew Zisserman

In Information Extraction

  1. Giovanni Maria Farinella
  2. Antonino Furnari
  3. Francesco Ragusa
  4. Spyridon Gidaris
  5. Nikos Komodakis
  6. Hiroki Ohashi
  7. Masato Tamura
  8. Chen Gao

Citations in the corpus