Recognizing Materials Using Perceptually Inspired Features

By Lavanya Sharan, Ce Liu, Ruth Rosenholtz, Edward H. Adelson. 2013

202 citations recorded by the source corpus.

Our world consists not only of objects and scenes but also of materials of various kinds. Being able to recognize the materials that surround us (e.g., plastic, glass, concrete) is important for humans as well as for computer vision systems. Unfortunately, materials have received little attention in the visual recognition literature, and very few computer vision systems have been designed specifically to recognize materials. In this paper, we present a system for recognizing material categories from single images. We propose a set of low and mid-level image features that are based on studies of human material recognition, and we combine these features using an SVM classifier. Our system outperforms a state-of-the-art system (Varma and Zisserman, TPAMI 31(11):2032–2047, 2009) on a challenging database of real-world material categories (Sharan et al., J Vis 9(8):784–784a, 2009). When the performance of our system is compared directly to that of human observers, humans outperform our system quite easily. However, when we account for the local nature of our image features and the surface properties they measure (e.g., color, texture, local shape), our system rivals human performance. We suggest that future progress in material recognition will come from: (1) a deeper understanding of the role of non-local surface properties (e.g., extended highlights, object identity); and (2) efforts to model such non-local surface properties in images.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Describing Textures in the Wild (2013, methodology): “2): on KTH-TIPS2-b, the describable attributes yield 61.1% mean accuracy and 49.0% on FMD outperforming the aLDA model of [29] combining color, SIFT and edge-slice (44.6%).”
Visually Indicated Sounds (2015, background): “This task implicitly requires material recognition, but unlike traditional work on this problem [4, 38], we never explicitly tell the algorithm about materials.”
Material recognition in the wild with the Materials in Context Database (2014, methodology): “FMD has been used in research on new features and learning methods for material perception and recognition [17, 10, 20, 25].”
Discovering states and transformations in image collections (2015, background): “As a result, there are many great datasets that cover object [9, 7, 31, 25, 20], attribute [16, 1, 10], material [26], and scene categories [32, 33].”
Locally-Transferred Fisher Vectors for Texture Classification (2017, methodology): “While the earlier studies have mainly used the bag-of-words (BOW) model and its variations [14, 11, 34, 13, 30, 19, 18], encoding via Fisher vectors (FV) has become the dominant approach in texture classification [21, 6, 25, 7].”
Rigid-Motion Scattering for Texture Classification (2014, methodology): “UMD, 2009) both contains 25 classes of 40 samples with uncontrolled deformations including shear, perspectivity effects and Training size 50 SRP (Liu et al., 2011) 48:2 Best single feature (SIFT) in (Sharan et al., 2013) 41:2 Rigid-motion scattering + log on grey images 51:22 Rigid-motion scattering”
Deep filter banks for texture recognition and segmentation (2015, background): “While this combination is already significantly above the state-of-the-art in DTD and FMD (+2.6%/11.2%), the method of [7] still outperforms these descriptors in KTH.”
From BoW to CNN: Two Decades of Texture Representation for Texture Classification (2018, methodology): “The Flickr Material Database (FMD) [206, 207] was built to address some of these limitations, by collecting many different object instances from the Internet grouped in 10 different material categories, with examples shown in Fig.”
Pairwise Rotation Invariant Co-Occurrence Local Binary Pattern (2014, methodology): “[48] extended [47] by using a kernel SVM and reported that SVM outperforms the aLDA classifier significantly.”
WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation (2023, background): “While the major works in computer vision focus on object, scene, or material recognition [13, 34, 38, 44], state classification aims to differentiate the fine-grained sub-object physical properties or attributes.”
Deep convolutional filter banks for texture recognition and segmentation (2014, background): “The ﬁrst one is the Flickr Material Dataset (FMD) [35], a recent benchmark containing 10 material classes.”
Face Anti-Spoofing with Human Material Perception (2020, methodology): “To distinguish live faces from spoofing faces, at the training stage, we normalize live depth maps and spoofing reflection maps in a range of [0, 1], while setting spoofing depth maps and live reflection maps to 0, which is similar to [14,28].”