iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection

By Chen Gao, Yuliang Zou, Jia-Bin Huang. British Machine Vision Conference, 2018

335 citations recorded by the source corpus.

Recent years have witnessed rapid progress in detecting and recognizing individual object instances. To understand the situation in a scene, however, computers need to recognize how humans interact with surrounding objects. In this paper, we tackle the challenging task of detecting human-object interactions (HOI). Our core idea is that the appearance of a person or an object instance contains informative cues on which relevant parts of an image to attend to for facilitating interaction prediction. To exploit these cues, we propose an instance-centric attention module that learns to dynamically highlight regions in an image conditioned on the appearance of each instance. Such an attention-based network allows us to selectively aggregate features relevant for recognizing HOIs. We validate the efficacy of the proposed network on the Verb in COCO and HICO-DET datasets and show that our approach compares favorably with the state-of-the-arts.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/72976d066d38d3d378d75dcf1467b0a295acad6b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

HOTR: End-to-End Human-Object Interaction Detection with Transformers (2021, background): “Therefore, most of the studies that have shown high performance in Rare settings make use of additional information, such as spatial layouts [5], pose information [18], linguistic priors [17], and coherence patterns between the humans and objects [16].”
The Open Images Dataset V4 (2018, background): “Recently, high-performing models based on deep convolutional neural networks are dominating the field (Gupta and Malik, 2015; Gkioxari et al., 2018; Gao et al., 2018; Kolesnikov et al., 2018).”
End-to-End Human Object Interaction Detection with HOI Transformer (2021, background): “Both human stream and object stream usually encode visual features for human and object boxes respectively [7].”
HAKE: Human Activity Knowledge Engine (2019, background): “HOI [12, 23, 21] usually accounts for a large proportion of daily life activities.”
Video-Text Pre-training with Learned Regions (2021, methodology): “To model the dynamic motion of objects, some recent works [20, 22, 45, 74] extract the instance-centric features from a video according to the ground-truth or de-”