Improved Multimodal Deep Learning with Variation of Information

By Kihyuk Sohn, Wenling Shang, Honglak Lee. 2014

191 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Multimodal Learning

Citations in the corpus

Generative Adversarial Text to Image Synthesis (2016, background): “Sohn et al. (2014) proposed a multimodal conditional prediction framework (hallucinating one modality given the other) and provided theoretical justification.”
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction (2016, methodology): “Our method is especially related to [4, 5, 41], which use bidirectional data prediction to learn representations from two sensory modalities.”
Attribute2Image: Conditional Image Generation from Visual Attributes (2015, background): “Our work can be considered in the context of multimodal learning (e.g., [29, 35, 34]).”
Learning Deep Representations of Fine-Grained Visual Descriptions (2016, background): “In [42], a novel information theoretic objective is developed, improving the performance of deep multimodal learning for images and text.”
Deep Multimodal Learning: A Survey on Recent Advances and Trends (2017, background): “For example, Sohn et al. [66] proposed a cost function that minimized the variation of information between modalities to learn relationships between modalities.”
Learning Factorized Multimodal Representations (2018, methodology): “…representation learning captures the interactions between modalities by modeling the joint distribution P ( X 1 , ⋯ , X M ) using either undirected graphical models (Srivastava & Salakhutdinov, 2012), directed graphical models (Suzuki et al., 2016b), or neural networks (Sohn et al., 2014).”
A Survey of Multi-View Representation Learning (2016, unknown): “[167] investigate an improved multi-modal RBM framework via minimizing the variation of information between data modalities through the shared latent representation.”
Multi-view Recurrent Neural Acoustic Word Embeddings (2016, background): “While many deep multi-view learning objectives are applicable (Ngiam et al., 2011; Srivastava & Salakhutdinov, 2014; Sohn et al., 2014; Wang et al., 2015), we consider the multi-view contrastive loss objective of (Hermann & Blunsom, 2014), which is simple to optimize and implement and performs well…”
Think Locally, Act Globally: Federated Learning with Local and Global Representations (2020, background): “Representation Learning involves learning informative features from data that are useful for generative [69, 71, 68] and discriminative [48, 47, 74, 58] tasks.”
Deep Variational Canonical Correlation Analysis (2016, methodology): “We use the same image and text features as in previous work (Srivastava & Salakhutdinov, 2014; Sohn et al., 2014): the image feature vector is a 3857-dimensional real-valued vector of handcrafted features, while the text feature vector is a 2000-dimensional binary vector of frequent tags.”
deepTarget: End-to-end Learning Framework for microRNA Target Prediction using Deep Recurrent Neural Networks (2016, unknown): “To resolve this issue, deep learning methods have been proposed to learn joint representations that are shared across multiple modalities after learning modality-specific network layers [42].”
A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data (2014, methodology): “The work of Srivastava and Salakhutdinov [12] on DBMs and Sohn et al. [15] on MDRNN are good recent examples.”
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities (2018, unknown): “[52] 87 trained Restricted Boltzmann Machines to minimize the variation of information between modality88 specific latent variables.”
On Deep Multi-View Representation Learning (2015, background): “There has also been work on multi-view feature learning using deep Boltzmann machines (Srivastava and Salakhutdinov, 2014; Sohn et al., 2014).”
Learning Structured Output Representation using Deep Conditional Generative Models (2015, methodology): “Such a recurrent connection has been applied for structured output prediction problems [23, 13, 28] to sequentially update the prediction by revising the previous guess while effectively deepening the convolutional network.”
Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks (2015, background): “Compared with [32], our objective is to learn and use dimension-wise feature relationships.”
Learning Robust Representations via Multi-View Information Bottleneck (2020, background): “Training images with less than two tags are removed, which reduces the total number of training samples to 749,647 pairs (Sohn et al., 2014; Wang et al., 2016).”