Extending Long Short-Term Memory for Multi-View Structured Learning

By Shyam Sundar Rajagopalan, Louis-philippe Morency, Tadas Baltrusaitis, Roland Göcke. 2016

128 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Model Architectures

Citations in the corpus

Multimodal Language Analysis with Recurrent Multistage Fusion (2018, background): “At its core, these multimodal signals are Recurrent Multistage Fusion highly structured with two prime forms of interactions: intra-modal and cross-modal interactions (Rajagopalan et al., 2016).”
Memory Fusion Network for Multi-view Sequential Learning (2018, background): “Recently, multi-view LSTM models have been proposed for multimodal setups where the LSTM memory is partitioned into different components for different views (Rajagopalan et al. 2016).”
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis (2020, methodology): “egorized into utterance-level and interutterance contextual models. Utterance-level baselines include: •Networks which perform temporal modeling and fusion of utterances: MFN [59], MARN [61], MV-LSTM [46], RMFN [24]. •Models which utilize attention and transformer modules to improvetokenrepresentati”
Multi-attention Recurrent Network for Human Communication Comprehension (2018, background): “The best performing model is reported as EFLSTM(⋆), ⋆ ∈ {-, s, b, sb} denoting vanilla, stacked, bidirectional and stacked bidirectional LSTMs respectively.”
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors (2018, methodology): “Such methods usually perform fusion throughout the multimodal sequence (Rajagopalan et al. 2016; Liang, Zadeh, and Morency 2018), leading to explicit modeling of both modality-speciﬁc and cross-modal interactions at every time step.”
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph (2018, background): “This complexity is rooted in variability of intra-modal and cross-modal dynamics for language, vision and acoustic modalities (Rajagopalan et al., 2016).”
Multimodal Machine Learning: A Survey and Taxonomy (2017, background): “representing audio-visual data for affect recognition [37], [152] and to represent multi-view data such as different visual cues for human behavior analysis [166].”
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors (2018, methodology): “Multi-View LSTM Multi-View LSTM (MV-LSTM) (Rajagopalan et al., 2016) aims to capture both modality-speciﬁc and cross-modality interactions from multiple modalities by partitioning the memory cell and the gates corresponding to multiple modalities.”
Factorized Multimodal Transformer for Multimodal Sequential Learning (2019, methodology): “Multiple FMS can be used inside a MTL to efﬁciently extract diverse multimodal Model \ Metric BA F1 MAE Corr MV-LSTM (Rajagopalan et al., 2016) 73.9/– 74.0/– 1.019 0 .601 TFN (Zadeh et al., 2017) 73.9/– 73.4/– 1.040 0 .633 MARN (Zadeh et al., 2018b) 77.1/– 77.0/– 0.968 0.625 MFN (Zadeh et al., 2018a”
Sentiment Analysis using Imperfect Views from Spoken Language and Acoustic Modalities (2018, background): “…from these individual different views add richness to the sentiment classiﬁcation, the intra and inter view-interactions play an important role in better sentiment classiﬁcation (Zadeh et al., 2017; Chen et al., 2018; Rajagopalan et al., 2016; Nojavanasghari et al., 2016; Xu et al., 2013).”
Learning Factorized Multimodal Representations (2018, unknown): “For a detailed description of the baselines, we point the reader to MFN (Zadeh et al., 2018a), MARN (Zadeh et al., 2018b), TFN (Zadeh et al., 2017), BC-LSTM (Poria et al., 2017), MV-LSTM (Rajagopalan et al., 2016), EF-LSTM (Hochreiter & Schmidhuber, 1997; Graves et al., 2013; Schuster & Paliwal, 199”