A Model for Learning the Semantics of Pictures

By Victor Lavrenko, Raghavan Manmatha, Jiwoon Jeon. 2003

809 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics (2013, neutral): “…or tags (see, e.g., Blei & Jordan, 2003; Barnard, Duygulu, Forsyth, Freitas, Blei, & Jordan, 2003; Feng & Lapata, 2008; Deschacht & Moens, 2007; Lavrenko, Manmatha, & Jeon, 2004; Makadia, Pavlovic, & Kumar, 2010; Weston, Bengio, & Usunier, 2010), but the much harder problem of automatically…”
Microsoft COCO Captions: Data Collection and Evaluation Server (2015, background): “The automatic generation of captions for images is a long standing and challenging problem in artiﬁcial intelligence [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19].”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, methodology): “We apply sparse SVD (Larsen, 1998) to the tag feature T to obtain a low-rank decomposition as T = U1SU>2 .”
Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers (2008, methodology): “Many learning approaches have been used for annotating images which include translation models [5], statistical models [2, 6], classification approaches [7–9] and relevance language models [10, 11].”
XRCE ’ s participation to ImagEval (2007, background): “In [21], they showed that working directly with continuous features describing the blobs instead of quantizing them into clusters (which is the case of CMRM, co-occurrence and translation models) performs better.”
I know what you did last summer: object-level auto-annotation of holiday snaps (2009, background): “In general, auto-annotation is one of the most relevant applications for object recognition methods, and accordingly a large amount of work has been devoted to that problem [2] [4] [5] [11] [12] [19] [23] [24] [25].”
A new approach to cross-modal multimedia retrieval (2010, background): “In computer vision, substantial effort has been devoted to the problem of image annotation [4, 12, 9, 16, 19, 1].”
Multiple Bernoulli relevance models for image and video annotation (2004, positive): “Our model is called Multiple-Bernoulli Relevance Model (MBRM) and is based on the Continuous-space Relevance Model (CRM) proposed by [8].”
Baselines for Image Annotation (2010, methodology): “The top portion of the table displays published results from several top-performing methods that approach the annotation problem from different perspectives, using different image representations: CRM (Lavrenko et al. 2004), InfNet (Metzler and Manmatha 2005), NPDE (Yavlinsky et al. 2005), MBRM (Fen”
Visual Contextual Advertising: Bringing Textual Advertisements to Images (2010, background): “In (Jeon et al. 2003), continuousspace relevance modelwas proposed to better handle continues features and be free from the influence of image blobs clustering.”
Automatic tag expansion using visual similarity for photo sharing websites (2010, background): “Recent studies in automatic annotation of visual data is promising [1–3, 5– 7, 9, 12, 14, 20, 23, 24, 26, 35, 36].”
Translating topics to words for image annotation (2007, neutral): “In the second category of approaches [5, 4, 8, 9].”
Topic Models for Image Annotation and Text Illustration (2010, methodology): “In Feng and Lapata (2008) we addressed this problem with a modiﬁed version of the continuous relevance annotation model (Lavrenko et al., 2003).”
On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval (2014, background): “For example, in [29] a query text, and in [30] a query image is used to retrieve similar text documents and images, based on low-level text (e.g., words) and image (e.g., DCTs) representations, respectively.”
Automatic Image Annotation Using Auxiliary Text Information (2008, methodology): “Incidentally, LDA can be also used to rerank the output of Lavrenko et al.’s (2003) model.”
Supervised Learning of Semantic Classes for Image Annotation and Retrieval (2007, methodology): “The various methods differ in the definition of the states of the hidden variable: Some associate a state to each image in the database [13], [21], while others associate them with image clusters [3], [12], and some model higher-level groupings, e.”
How Many Words Is a Picture Worth? Automatic Caption Generation for News Images (2010, background): “…(Vailaya et al., 2001; Smeulders et al., 2000), instantiations of the noisychannel model (Duygulu et al., 2002), latent variable models (Blei and Jordan, 2003; Barnard et al., 2002; Wang et al., 2009), and models inspired by information retrieval (Lavrenko et al., 2003; Feng et al., 2004).”
Image retrieval: Ideas, influences, and trends of the new age (2008, methodology): “Cross-Media relevance models models have been used for image annotation in [Jeon et al. 2003; Lavrenko et al. 2003].”
Topic regression multi-modal Latent Dirichlet Allocation for image annotation (2010, background): “Several modeling variations have been proposed, see [1] [2] [9] [8], with the different forms of probability distribution assumed for caption words (multinomial vs. bernoulli) and image features (mixture of Gaussian vs. non-parameteric density estimation).”