Start from Scratch: Towards Automatically Identifying, Modeling, and Naming Visual Attributes

By Hanwang Zhang, Yang Yang, Huanbo Luan, Shuicheng Yan, Tat-seng Chua. 2014

83 citations recorded by the source corpus.

Higher-level semantics such as visual attributes are crucial for fundamental multimedia applications. We present a novel attribute discovery approach that can automatically identify, model and name attributes from an arbitrary set of image and text pairs that can be easily gathered on the Web. Different from conventional attribute discovery methods, our approach does not rely on any pre-defined vocabularies and human labeling. Therefore, we are able to build a large visual knowledge base without any human efforts. The discovery is based on a novel deep architecture, named Independent Component Multimodal Autoencoder (ICMAE), that can continually learn shared higher-level representations across the visual and textual modalities. With the help of the resultant representations encoding strong visual and semantic evidences, we propose to (a) identify attributes and their corresponding high-quality training images, (b) iteratively model them with maximum compactness and comprehensiveness, and (c) name the attribute models with human understandable words. To date, the proposed system has discovered 1,898 attributes over 1.3 million pairs of image and text. Extensive experiments on various real-world multimedia datasets demonstrate the quality and effectiveness of the discovered attributes, facilitating multimedia applications such as image annotation and retrieval as compared to the state-of-the-art approaches.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Xiangnan He
2. Tat-seng Chua
3. Liqiang Nie
4. Hanwang Zhang
5. Dacheng Tao
6. Luc Van Gool
7. Deyu Meng
8. Yi Yang

Citations in the corpus

• Neural Collaborative Filtering (2017, methodology): “The neural network has been proven to be capable of approximating any continuous function [17], and more recently deep neural networks (DNNs) have been found to be effective in several domains, ranging from computer vision, speech recognition, to text processing [5, 10, 15, 47].”
• Cross-Modality Person Re-Identification with Generative Adversarial Training (2018, methodology): “Representative methods include, but not limited to, traditional statistical correlation analysis, DNN-based methods [Zhang et al. , 2014], cross-media graph regularization methods, metric learning methods, and dictionary learning methods.”
• Text-to-image Synthesis via Symmetrical Distillation Networks (2018, unknown): “As a multimedia technology, cross-modal retrieval [6, 20, 29, 38, 43] has become a highlighted topic of research in the big data era.”
• Self-Paced Curriculum Learning (2015, background): “2014b), by exploiting semantic features (Tan, Jiang, and Neo 2014; Liu et al. 2013; Zhang et al. 2014; Inoue and Shinoda 2014; Jiang, Hauptmann, and Xiang 2012; Tang et al. 2012; Yu, Jiang, and Hauptmann 2014; Cao et al. 2013), and by automatic speech recognition (Miao, Metze, and Rawat 2013; Miao e”
• An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges (2017, background): “Some works also consist of two autoencoders, such as independent component multimodal autoencoder (ICMAE) [37] and deep canonically correlated autoencoders (DCCAE) [38].”