Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach

By Jia Li, James Ze Wang. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003

1,222 citations recorded by the source corpus.

Automatic linguistic indexing of pictures is an important but highly challenging problem for researchers in computer vision and content-based image retrieval. In this paper, we introduce a statistical modeling approach to this problem. Categorized images are used to train a dictionary of hundreds of statistical models each representing a concept. Images of any given concept are regarded as instances of a stochastic process that characterizes the concept. To measure the extent of association between an image and the textual description of a concept, the likelihood of the occurrence of the image based on the characterizing stochastic process is computed. A high likelihood indicates a strong association. In our experimental implementation, we focus on a particular group of stochastic processes, that is, the two-dimensional multiresolution hidden Markov models (2D MHMMs). We implemented and tested our ALIP (Automatic Linguistic Indexing of Pictures) system on a photographic image database of 600 different concepts, each with about 40 training images. The system is evaluated quantitatively using more than 4,600 images outside the training database and compared with a random annotation scheme. Experiments have demonstrated the good accuracy of the system and its high potential in linguistic indexing of photographic images.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f39d3e88cce063ccd3ca01100efd44dcabc9d3b4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards total scene understanding: Classification, annotation and segmentation in an automatic framework (2009, background): “Some go further in assigning a list of annotations without localizing where in the image each annotation belongs (e.g. [16]).”
XRCE ’ s participation to ImagEval (2007, methodology): “[22] J. Li and J. Z. Wang.”
CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval (2008, methodology): “Initially, experiments were performed in the database of 1000 images that was used by James Wang [2] [13].”
LifeLogging: Personal Big Data (2014, methodology): “Automatic recognition of events, where the event has some semantic significance, has been identified as important for many years in applications like managing personal photos, as described in Lim et al. (2003), indexing surveillance videos by Atrey et al.”
Animals on the Web (2006, background): “There has been extensive work on fusing the information available from these two modalities to perform various tasks such as clustering art [2], labeling images [1, 15, 12] or identifying faces in news photographs [6].”
MILES: Multiple-Instance Learning via Embedded Instance Selection (2006, background): “Far beyond the drug activity prediction problem, the multiple-instance problem emerges naturally in a variety of challenging learning problems in computer vision, including natural scene classification [37], content-based image retrieval [34], [56], [60], [15], image categorization [6], [32], [16], ”
Automatic multimedia cross-modal correlation discovery (2004, methodology): “For example, to associate words with images for automatic image captioning, people have proposed methods based on classiers [15] or information retrieval techniques (relevance model [10] and user feedback [26]), as well as building association models (translation model [7]; hierarchical model [2, 3,”
Image retrieval: Ideas, influences, and trends of the new age (2008, neutral): “… 2002], to involve every generated segment of an image in the matching process to obtain soft similarity measures [Wang et al. 2001], or to characterize spatial arrangement of color and texture using block-based 2D multiresolution hidden Markov models (MHMMs) [Li et al. 2000; Li and Wang 2003].”
Translating topics to words for image annotation (2007, methodology): “Existing techniques for automatic image annotation can be summarized into two categories [1, 6, 2].”
CLUE: cluster-based retrieval of images by unsupervised learning (2005, neutral): “Digital Object Identifier 10.1109/TIP.2005.849770”
Topic regression multi-modal Latent Dirichlet Allocation for image annotation (2010, background): “The first line of work considers a discriminative approach and cast the problem of automatic annotation as a classification problem, treating annotation words as class labels [6] [10].”
Content-based image retrieval: approaches and trends of the new age (2005, neutral): “In the case of image retrieval, some of the ways of getting around this problem have been to reduce dependence on reliable segmentation [12], to involve every generated segment of an image in the matching process to obtain soft similarity measures [104], or to characterize spatial arrangement of col”
A review on automatic image annotation techniques (2012, result): “in the same way as text documents [17–40,115,116].”
A New Baseline for Image Annotation (2008, methodology): “To address this problem, [19] developed a real-time ALIPR image search engine which uses multiresolution 2D Hidden Markov Models to model concepts determined by a training set.”
Connecting modalities: Semi-supervised segmentation and annotation of images using unaligned text corpora (2010, background): “com [15], CorrLDA [4] and Towards Total Scene Understanding [16].”
Automatic Caption Generation for News Images (2013, neutral): “, 2003), or averagely cut rectangles (Feng et al., 2004; Li and Wang, 2003; Wang and Li, 2002; Jeon and Manmatha, 2004)).”
Supervised Learning of Semantic Classes for Image Annotation and Retrieval (2007, methodology): “GMM-DCT outperformed the 2D-MHMM of [23] in all cases, with an improvement of about 0.”
Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers (2008, methodology): “Many learning approaches have been used for annotating images which include translation models [5], statistical models [2,6], classification approaches [7, 8 ,9] and relevance language models [10,11].”