Visual Information in Semantic Representation

By Yansong Feng, Mirella Lapata. 2010

123 citations recorded by the source corpus.

Subfields: Multimodal Learning, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Multimodal Learning

In Computational Linguistics & Semantics

Citations in the corpus

Zero-Shot Learning Through Cross-Modal Transfer (2013, background): “Some work has been done on multimodal distributional methods [11, 23].”
Multimodal Distributional Semantics (2013, methodology): “Table 3: Pearson correlation of some of our best multimodal combinations on the WordSim subset covered by Feng and Lapata (2010) (all coefficients significant with p < 0.001; Pearson used instead of Spearman for full comparability with Feng and Lapata). The models assigned 0 similarity to the 71/253”
Combining Language and Vision with a Multimodal Skip-gram Model (2015, background): “…followed by fusion), and they are only tested on concepts for which both textual and visual labeled training data are available (the pioneering model of Feng and Lapata (2010) did learn from text and images jointly using Topic Models, but was shown to be empirically weak by Bruni et al. (2014)).”
Going Beyond Text: A Hybrid Image-Text Approach for Measuring Word Relatedness (2011, methodology): “The research that is most closely related to ours is the work of (Feng and Lapata, 2010), where it has been shown that it is possible to combine visual representations of word meanings into a joint bimodal representation constructed by using probabilistic generative latent topic models.”
Vector Space Models of Word Meaning and Phrase Meaning: A Survey (2012, background): “Under a topic model, a word can be characterized as a point in a vector space in which the dimensions are topics, and a word’s coordinates are its probabilities under all the topics (e.g., Feng and Lapata 2010; Dinu and Lapata 2010).”
Multimodal Classification Fusion in Real-World Scenarios (2017, unknown): “compositionality and classification [2], [3], [4], [5], [6].”
A Survey of Multi-View Representation Learning (2016, unknown): “Nguyen et al. [119] propose a Multi-modal Multi-instance Multi-Label LDA model (M3LDA), in which the model consists of a visual-label part, a textual-label part, and a label-topic part.”
Is this a wampimuk? Cross-modal mapping between distributional semantics and the visual world (2014, background): “Very recently, a number of papers have exploited advances in automated feature extraction form images and videos to enrich context vectors with visual information (Bruni et al., 2014; Feng and Lapata, 2010; Leong and Mihalcea, 2011; Regneri et al., 2013; Silberer et al., 2013).”
Learning Grounded Meaning Representations with Autoencoders (2014, background): “…by feature norms elicited from humans (Andrews et al., 2009; Steyvers, 2010; Silberer and Lapata, 2012), visual information extracted automatically from images, (Feng and Lapata, 2010; Bruni et al., 2012a; Silberer et al., 2013) or a combination of both (Roller and Schulte im Walde, 2013).”
Formal Distributional Semantics: Introduction to the Special Issue (2016, background): “Recently, researchers have started exploring other modalities, using, for instance, visual and auditory information extracted from images and sound ﬁles (Feng and Lapata 2010; Bruni et al. 2012; Roller and Schulte Im Walde 2013; Kiela and Clark 2015; Lopopolo and van Miltenburg 2015).”
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech (2017, methodology): “such as word similarity [57]–[59], word association [60], [61] or concept categorisation [62].”
Learning Image Embeddings using Convolutional Neural Networks for Improved Multi-Modal Semantics (2014, methodology): “We conjecture that the performance increase on WordSim353 is due to increased performance on the similarity subset of that dataset.”
Distributional Models of Word Meaning (2018, background): “…analyzed in this review, further research issues in which DS is producing very interesting results include (a) the development of multimodal DSMs (Feng & Lapata 2010, Bruni et al. 2014) that integrate corpus-derived features and features extracted from images, which are also used to explore the…”
Distributional Semantics in Technicolor (2012, background): “Recently, NLPers have begun exploiting BoVW to enrich distributional models that represent word meaning with visual features automatically extracted from images (Feng and Lapata, 2010; Bruni et al., 2011; Leong and Mihalcea, 2011).”
Automatic Caption Generation for News Images (2013, background): “Humans can describe images effortlessly, probably because they have a common underlying representation for the two modalities (Feng and Lapata, 2010c).”
Multimodal Machine Learning: A Survey and Taxonomy (2017, methodology): “Starting from work by Feng and Lapata [59], grounding is usually performed by finding a common latent space between the representations [59], [183] (in case of parallel datasets) or by learning unimodal representations separately and then concatenating them to lead to a multimodal one [29], [101], [”
Grounded Models of Semantic Representation (2012, background): “For example, Feng and Lapata (2010) learn semantic representations from cor-”