A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping

By Wenguan Wang, Jianbing Shen, Haibin Ling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

322 citations recorded by the source corpus.

We study the problem of photo cropping, which aims to find a cropping window of an input image to preserve as much as possible its important parts while being aesthetically pleasant. Seeking a deep learning-based solution, we design a neural network that has two branches for attention box prediction (ABP) and aesthetics assessment (AA), respectively. Given the input image, the ABP network predicts an attention bounding box as an initial minimum cropping window, around which a set of cropping candidates are generated with little loss of important information. Then, the AA network is employed to select the final cropping window with the best aesthetic quality among the candidates. The two sub-networks are designed to share the same full-image convolutional feature map, and thus are computationally efficient. By leveraging attention prediction and aesthetics assessment, the cropping model produces high-quality cropping results, even with the limited availability of training data for photo cropping. The experimental results on benchmark datasets clearly validate the effectiveness of the proposed approach. In addition, our approach runs at 5 fps, outperforming most previous solutions. The code and results are available at: https://github.com/shenjianbing/DeepCropping.

Subfields: Computer Vision and Pattern Recognition, Sensory Systems

View Paper: https://www.semanticscholar.org/paper/55ba76bdca99a1ab07af91e0ebde0bf595d71652

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Sensory Systems

Citations in the corpus

Salient Object Detection in the Deep Learning Era: An In-Depth Survey (2019, methodology): “In computer graphics, SOD has been used tasks such as non-photo-realist rendering [15], [16], automatic image cropping [17], image retargeting [18], [19], video summarization [20], [21], etc .”
SwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection (2022, background): “3127149 cropping [29], [30], retargeting [31], quality assessment [32]”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, background): “The effectiveness of the attention methods [1], [2] or attention modules [23], [24], [25] beneﬁt a lot of vision tasks [26], [27], [28], [29], [30], [31], [32].”
Revisiting Video Saliency Prediction in the Deep Learning Era (2021, background): “Recently, deep learning based static saliency models [28], [29], [30], [53], [54], [55], [56] have achieved great improvements, relying on the powerful end-to-end learning ability of neural networks and the availability of large-scale static saliency data-sets [2].”
Salient Object Detection With Pyramid Attention and Salient Edges (2019, background): “It can be widely applied for improving a variety of vision tasks, such as object proposal generation [2], object segmentation [42, 44], photo cropping [45, 41], and video object tracking [13], among others.”