Recognizing Image Style

By Sergey Karayev, Matthew Trentacoste, Helen Han, Aseem Agarwala, Trevor Darrell, Aaron Hertzmann, Holger Winnemoeller. 2013

498 citations recorded by the source corpus.

The style of an image plays a significant role in how it is viewed, but style has received little attention in computer vision research. We describe an approach to predicting style of images, and perform a thorough evaluation of different image features for these tasks. We find that features learned in a multi-layer network generally perform best -- even when trained with object class (not style) labels. Our large-scale learning methods results in the best published performance on an existing dataset of aesthetic ratings and photographic style annotations. We present two novel datasets: 80K Flickr photographs annotated with 20 curated style labels, and 85K paintings annotated with 25 style/genre labels. Our approach shows excellent classification performance on both datasets. We use the learned classifiers to extend traditional tag-based image search to consider stylistic constraints, and demonstrate cross-dataset understanding of style.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Caffe: Convolutional Architecture for Fast Feature Embedding (2014, background): “Members of Berkeley EECS have also collaborated with several industry partners such as Facebook [11] and Adobe [6], using Caﬀe or its direct precursor (Decaf) to obtain state-of-the-art results.”
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models (2022, methodology): “In 5-shot settings, the gap is minor, which is similar to observations made for supervised CNNs [71, 25, 26].”
A Neural Algorithm of Artistic Style (2015, background): “20 There, classiﬁers are trained on top of the raw network activations, which we call content representations.”
Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering (2016, background): “, ‘Deep CNN’) features extracted from raw product images present as a good option due to their widely demonstrated efficacy at capturing abstract notions of fine-grained categories [31], photographic style [17], aesthetic quality [24], and scene characteristics [8], among others.”
VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback (2015, background): “Recently, high-level visual features from Deep Convolutional Neural Networks (‘Deep CNNs’) have seen successes in tasks like object detection (Rus-sakovsky et al., 2014), photographic style annotations (Karayev et al., 2014), and aesthetic quality categorization (Lu et al., 2014), among others.”
C3D: Generic Features for Video Analysis (2014, background): “These features are normally the activations of the network’s last few fully con-1 nected layers which are showed to be useful for transfer learning tasks [22, 54, 56].”
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views (2015, neutral): “However, this is contrary to the recent finding — features learned by task-specific supervision leads to much better task performance [16, 11, 14].”
DeepEdge: A multi-scale bifurcated deep network for top-down contour detection (2014, methodology): “e task of object class recognition, they have been shown to be highly effective for other image-analysis tasks, including object detection [11], attribute prediction [29], and image style recognition [14]. The network was trained on 1:2 million images and it includes more than 60 million parameters.”
Understanding Deep Features with Computer-Generated Imagery (2015, background): “These questions are important as CNNs with different architectures and trained/ﬁne tuned for different tasks have shown to perform differently [17, 40] or have different feature response characteristics [39].”
Slow and Steady Feature Analysis: Higher Order Temporal Coherence in Video (2015, background): “Comparison to supervised pretraining and finetuning: Recently, a two-stage supervised pretraining and finetuning strategy (SUP-FT) has emerged as the leading approach to solve visual recognition problems with limited training data where high-capacity models like deep neural networks may not be direc”
Universal Style Transfer via Feature Transforms (2017, methodology): “We randomly select 10 content images from [22] and 40 style images from [17], compute the averaged difference over all styles, and show the results in Table 2 (1st row).”
Image Aesthetic Assessment: An experimental survey (2016, methodology): “Fine-tuning from trained CNN has been proved in [61], [62] to be an effective initialization approach.”
Understanding and Creating Art with AI: Review and Outlook (2021, methodology): “trained on ImageNet [35], a large hand-labelled object dataset, as features for artistic style classification [70].”
Deep Classifiers from Image Tags in the Wild (2015, background): “Some of these missing tags are covered by scene [22, 30, 31] and style databases [11, 18].”
Wide-Area Image Geolocalization with Aerial Reference Imagery (2015, background): “(1) Intuitively, the objective is to learn to extract features from the aerial imagery that match those from a corresponding ground-level image.”