Gaze-based interaction for semi-automatic photo cropping

By Anthony Santella, Maneesh Agrawala, D. DeCarlo, David Salesin, Michael F. Cohen. International Conference on Human Factors in Computing Systems, 2006

400 citations recorded by the source corpus.

We present an interactive method for cropping photographs given minimal information about important content location, provided by eye tracking. Cropping is formulated in a general optimization framework that facilitates adding new composition rules, and adapting the system to particular applications. Our system uses fixation data</ to identify important image content and compute the best crop for any given aspect ratio or size, enabling applications such as automatic snapshot recomposition, adaptive documents, and thumbnailing. We validate our approach with studies in which users compare our crops to ones produced by hand and by a completely automatic approach. Experiments show that viewers prefer our gaze-based crops to uncropped images and fully automatic crops.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/54ae9ed45bd1f567d62f51421a1b8d66768bb756

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking (2015, background): “An understanding of human visual attention is essential to many applications in computer vision, computer graphics, computational photography, psychology, sociology, and human-computer interaction [31, 12].”
Saliency Detection by Multitask Sparsity Pursuit (2012, background): “Saliency detection is related to many applications, such as automatic image cropping [5], image thumbnailing [6], image/video compressing [7], and image collection browsing [8].”
Saliency in VR: How Do People Explore Virtual Environments? (2016, background): “Note that this approach to thumbnail generation is also closely related to techniques for gaze-based photo cropping [48].”
We get the algorithms of our ground truths: Designing referential databases in digital image processing (2017, background): “8 At that time, if the task-independent and adaptive character of saliency detection was theoretically interesting for automatic image cropping (Santella et al., 2006), adaptive display on small devices (Chen et al.”
Learning to Detect A Salient Object (2007, background): “There are many applications for visual attention, for example, automatic image cropping [23], adaptive image display on small devices [4], image/video compression, advertising design [7], and image collection browsing.”
Learning to predict where humans look (2009, background): “For example, an understanding of visual attention is useful for automatic image cropping [16], thumbnailing, or image search.”
What Do Different Evaluation Metrics Tell Us About Saliency Models? (2016, background): “[54], [87], thumbnailing [55], [76], content-aware image re-targeting and cropping [3], [4], [69], [72], [85], rendering and visualization [42], [52], collage [31], [86] and artistic rendering [20], [41] require ranking (by importance or saliency) different image regions.”