Dynamic Eye Movement Datasets and Learnt Saliency Models for Visual Action Recognition

By Stefan Mathe, Cristian Sminchisescu. 2012

163 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Delving into Egocentric Actions (2015, neutral): “Most recently, Mathe and Sminchesescu [23] demonstrate promising results for recognizing actions by sampling local descriptors from a predicted saliency map.”
Finding action tubes (2014, methodology): “One could use more complicated techniques, such as action saliency detectors trained on human eye fixations and low level cues [29].”
Saliency-aware geodesic video object segmentation (2015, background): “These saliency methods [14, 20, 28, 26, 13, 21], however, usually build their system as a simple combination of existing image saliency models with motion cues.”
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos (2021, background): “These include attention mechanisms from gaze and motor attention [47, 38, 42], active object detection [20, 7, 15, 70], and hands in contact [66, 60, 33].”
Action and Event Recognition with Fisher Vectors on a Compact Feature Set (2013, unknown): “Mathe and Sminchisescu [24] use multiple-kernel learning to combine 14 descriptors sampled on human attention maps with the dense trajectory features of [41].”
LEAR-INRIA submission for the THUMOS workshop (2013, result): “This results in similar effects as sampling features based on visual saliency maps [6, 12].”
Space-Variant Descriptor Sampling for Action Recognition Based on Saliency and Eye Movements (2012, background): “For alternative work, published in this conference, on eye movement data collection and saliency models for action recognition see [15].”
Eye tracking assisted extraction of attentionally important objects from videos (2015, methodology): “The proposed technique outperforms state-of-the-art video segmentation using eye tracking prior and obtains favorable object extraction over algorithms which do not utilize eye tracking data.”
Training Object Class Detectors from Eye Tracking Data (2014, background): “Several authors have collected eye tracking data for video and shown that saliency maps computed from these data can improve action recognition [31,46].”
Attentive Systems: A Survey (2017, background): “Many works (Mathe and Sminchisescu 2012, 2013; Nguyen et al. 2015; Vig et al. 2012) investigate the benefit of space-variant processing of inputs, inspired by attentional mechanisms in the human visual system.”
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video (2018, methodology): “There have been several recent works that use human gaze for action recognition [24, 31, 7].”
Action from Still Image Dataset and Inverse Optimal Control to Learn Task Specific Visual Scanpaths (2013, methodology): “Human gaze pattern annotations have been collected for both static images[11, 13, 14, 12, 26, 18] and for video[19, 23, 15], see [24] for a recent overview.”
Y Qualification Main task Y Return N N Y User qualified ? N Feedback Pass ? Pass ? Annotator training Annotating images (2017, methodology): “Such data have been used as a weak supervisory signal to localize objects on images [41, 45] or videos [57, 40].”