Describing Textures in the Wild

By Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Shakir Mohamed, Andrea Vedaldi. 2013

3,677 citations recorded by the source corpus.

Patterns and textures are key characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly. Aiming at supporting this dimension in image understanding, we address the problem of describing textures with semantic attributes. We identify a vocabulary of forty-seven texture terms and use them to describe a large dataset of patterns collected "in the wild". The resulting Describable Textures Dataset (DTD) is a basis to seek the best representation for recognizing describable texture attributes in images. We port from object recognition to texture recognition the Improved Fisher Vector (IFV) and Deep Convolutional-network Activation Features (DeCAF), and show that surprisingly, they both outperform specialized texture descriptors not only on our problem, but also in established material recognition datasets. We also show that our describable attributes are excellent texture descriptors, transferring between datasets and tasks, in particular, combined with IFV and DeCAF, they significantly outperform the state-of-the-art by more than 10% on both FMD and KTH-TIPS-2b benchmarks. We also demonstrate that they produce intuitive descriptions of materials and Internet images.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Constructive Prediction of the Generalization Error Across Scales (2019, background): “DTD (Cimpoi et al., 2014): a texture database of 47 categories and 5640 images.”
Domain randomization for transferring deep neural networks from simulation to the real world (2017, methodology): “To examine how our detectors would perform on objects with complex textures, we created simulated scenes in which the object of interest is given a texture from the Describable Textures Dataset (DTD) [4].”
Do Better ImageNet Models Transfer Better? (2018, background): “These datasets covered a wide range of image classification tasks, including superordinate-level object classification (CIFAR-10 [37], CIFAR-100 [37], PASCAL VOC 2007 [19], Caltech-101 [20]); fine-grained object classification (Food-101 [5], Birdsnap [4], Stanford Cars [36], FGVC Aircraft [48], Oxfo”
On Robustness and Transferability of Convolutional Neural Networks (2020, neutral): “CIFAR-100 [38], DTD [9], Flowers102 [47], Pets [49], Sun397 [68], SVHN [46].”
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, background): “DTD (Cimpoi et al., 2014) The task consists in classifying images of textural patterns (47 classes, with 120 training images each).”
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, methodology): “[81] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi.”
Natural Adversarial Examples (2019, methodology): “For instance, previous work uses the Describable Textures Dataset [9], Places365 scenes [58], and synthetic blobs to test ImageNet OOD detectors.”
Fine-Grained Image Classification via Combining Vision and Language (2017, background): “This has been proved by a large amount of recognition tasks, such as object detection, texture recognition and fine-grained image classification [29, 30, 31, 32] etc.”
Unified Perceptual Parsing for Scene Understanding (2018, background): “Various image datasets are constructed only for speciﬁc task, such as ADE20K for scene parsing [2], the Describe Texture Dataset (DTD) for texture recognition [4], and OpenSurfaces for material and surface recognition [6].”
COCO-Stuff: Thing and Stuff Classes in Context (2016, background): “The more re- cent Describable Textures Dataset [12] instead collects textured patches in the wild, described by human-centric attributes.”
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP (2021, neutral): “, 2004), CIFAR10 (Krizhevsky, 2009), CIFAR100 (Krizhevsky, 2009), DTD (Cimpoi et al., 2014), Eurosat (Helber et al.”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “, 2009), Describable Textures (Cimpoi et al., 2014), Food-101 (Bossard et al.”
Learning to Prompt for Vision-Language Models (2021, neutral): “We observe that a few words are somewhat relevant to the tasks, such as “enjoyed” for Food101, “ﬂuﬀy” and “paw” for OxfordPets, and “pretty” for DTD.”
Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs (2021, neutral): “…Islam et al. [16] performed a large-scale case study on border heuristics, including padding and canvas colors, and showed that positional information can hurt (e.g., texture recognition [5]) or help (e.g., semantic segmentation and instance segmentation [32]) performance depending on the task.”
ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition (2021, background): “Meta-Dataset, a dataset of 10 datasets, formulates a more challenging task where whole datasets are held-out, but these datasets contain simple and clean images, such as clipart drawings of characters/symbols [23, 49, 17], and ImageNet-like images [26, 39, 32, 50, 6] showing objects in uniform light”