Deep Filter Banks for Texture Recognition, Description, and Segmentation

By Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Andrea Vedaldi. International Journal of Computer Vision, 2015

367 citations recorded by the source corpus.

Visual textures have played a key role in image understanding because they convey important semantics of images, and because texture representations that pool local image descriptors in an orderless manner have had a tremendous impact in diverse applications. In this paper we make several contributions to texture understanding. First, instead of focusing on texture instance and material category recognition, we propose a human-interpretable vocabulary of texture attributes to describe common texture patterns, complemented by a new describable texture dataset for benchmarking. Second, we look at the problem of recognizing materials and texture attributes in realistic imaging conditions, including when textures appear in clutter, developing corresponding benchmarks on top of the recently proposed OpenSurfaces dataset. Third, we revisit classic texture represenations, including bag-of-visual-words and the Fisher vectors, in the context of deep learning and show that these have excellent efficiency and generalization properties if the convolutional layers of a deep model are used as filter banks. We obtain in this manner state-of-the-art performance in numerous datasets well beyond textures, an efficient method to apply deep features to image regions, as well as benefit in transferring features from one domain to another.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b1e3b92007e426ee466f63c4df730d3a7a63dc2e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Bilinear CNNs for Fine-grained Visual Recognition (2015, methodology): “edu texture representations based on SIFT [40] and their recent extensions based on CNNs [11], [24], have been shown to be effective at fine-grained recognition, they have not matched the performance of part-based approaches.”
Image Quality Assessment: Unifying Structure and Texture Similarity (2020, methodology): “993 achieved by a data-driven CNN-based method [79].”
Deep learning for visual understanding (2017, positive): “In comparison with prior research [70, 170, 230, 233, 235, 236] which also attempted to incorporate BoW and CNN, BoSP has several di erences: First, BoSP does not need to generate the visual codebook, since the surrogate parts have already been inherently determined by the structure, i.”
On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location (2020, methodology): “The other approach to cropping regions is implicitly on featuremaps after feeding the full image to a CNN as done in Faster R-CNN [79], BagNet [8], and CNN pooling methods such as sum [6], BoW [76], VLAD [3, 28], Fisher vector [16].”
Visualizing and Understanding Deep Texture Representations (2015, methodology): “Using the Flickr Material Dataset (FMD) [34], Describable Texture Dataset (DTD) [3] and KTH-T2b [1] we show that it performs favorably to Fisher vector CNN model [4], which is the current state of the art.”
Bilinear CNN Models for Fine-Grained Visual Recognition (2015, methodology): “Recently, texture representations constructed from features corresponding to layers of a CNN have been shown to be very effective at fine-grained and texture recognition tasks [9], [22].”
Compact Bilinear Pooling (2015, methodology): “Fisher vector has an unsupervised dictionary learning phase, and it is unclear how to perform fine-tuning [8].”
Texture Feature Extraction Methods: A Survey (2019, background): “[248] have shown that deep learning can be used in many domains of texture analysis.”
From BoW to CNN: Two Decades of Texture Representation for Texture Classification (2018, methodology): “For approaches requiring a learned codebook, kmeans clustering [110, 114, 125, 237, 263] and Gaussian Mixture Models (GMM) [34, 36, 107, 91, 179, 209] are the most popular and successful methods.”
Deep features for breast cancer histopathological image classification (2017, methodology): “Often referred to as DeCAF features or neural codes, this approach has been previously applied to diverse tasks, such as object recognition [7], image retrieval [8], texture recognition [9], among others [10].”
UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory (2016, background): “normal estimation [2, 25, 103], depth estimation [25, 63, 64], intrinsic image decomposition [73] shadow detection [108], texture classification [19], material classification [4], saliency estimation [58, 112], semantic segmentation [14, 27,67], region proposal generation [30,82,85], instance segmen”