Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search

By Sung Ju Hwang, Kristen Grauman. 2012

130 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics (2013, methodology): “…data sets, but does not cause any problems here, since our training data consists of only 6,000 items ( KCCA has been successfully used to associate images (Hardoon et al., 2004; Hwang & Grauman, 2012; Hardoon et al., 2006) or image regions (Socher & Li, 2010) with individual words or sets of tags.”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, methodology): “C V ] 2 S ep 2 Several promising recent approaches for modeling images and associated text (Gong and Lazebnik, 2011; Hardoon et al., 2004; Hwang and Grauman, 2010, 2011; Rasiwasia et al., 2010) rely on canonical correlation analysis (CCA), a classic technique that maps two views, given by visual…”
Cross-Modal Retrieval With CNN Visual Features: A New Baseline (2017, result): “Specially, Hwang and Grauman [18] proposed an unsupervised learning method based on kernel CCA to discover the relationship between human tags and the relative importance of objects in the image.”
VIP: Finding important people in images (2015, methodology): “We model importance in two ways: • Image-Level Importance: “Given an image, who is the most important individual?””
Automatic Caption Generation for News Images (2013, background): “Examples include associating names mentioned in the captions to faces depicted in news images (e.g., [23], [24]), verbs to body poses [25], and learning models for recognizing objects [26] and their relative importance [27].”
Adopting Abstract Images for Semantic Scene Understanding (2016, background): “Interestingly, [39] and [4] found that the scale and location of an object is related to the order in which a user tags the object in an image.”
Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization (2014, background): “How to leverage such multimodal data to conduct cross-view similarity search (Rasiwasia et al. 2010; Hwang and Grauman 2012; Sharma et al. 2012; Gong et al. 2014) has become a challenging but interesting research problem.”
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics (Extended Abstract) (2015, methodology): “BoW1 is the Bag of Words baseline using 1 caption per training image, BoW5 is with all 5 captions, TagRank [Hwang and Grauman, 2012], TRI5 is the baseline using the subsequence string kernel [Shawe-Taylor and Cristianini, 2004], and TRI5SEM is our final model incorporating lexical semantics.”
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval (2020, methodology): “(ii) As paired-embedding is ignored in SoAs, shadowing cross-modal retrieval literature [11, 16, 43] we design a few Baselines (B) employing paired-embeddings: B-Siamese is a naive baseline built similar to TripletSN but replaces Sketch-a-Net with more advanced InceptionV3 as backbone feature extrac”
A Survey of Multi-View Representation Learning (2016, unknown): “As a non-linear extension of CCA, kernel CCA has been successfully applied in many situations, including independent component analysis [2], cross-media information retrieval [3, 84, 85], computational biology [3, 86, 87], multi-view clustering [46, 88], acoustic feature learning [89, 90], and stati”
Transductive Multi-view Embedding for Zero-Shot Recognition and Annotation (2014, background): “Most existing work [33, 13, 16, 36, 10, 9] focuses on modelling images/videos with associated text (e.”
Beyond Instance-Level Image Retrieval: Leveraging Captions to Learn a Global Visual Representation for Semantic Retrieval (2017, background): “Moving away from instances, some works have tackled visual search as the retrieval of images that share the same category label [7, 8] or a set of tags [29, 22].”