Deep Multimodal Learning: A Survey on Recent Advances and Trends

By Dhanesh Ramachandram, Graham W. Taylor. 2017

980 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion (2021, methodology): “The independent classifiers paradigm [65] coupled with the sigmoid activation and binary crossentropy loss exclude explicitly a late fusion [57] of the task embeddings resulting in more specialized classifiers.”
Deep Multimodal Representation Learning: A Survey (2019, background): “From the perspective of multimodal deep learning, the closest effort to ours is [20] which mainly reviews the models and applications relying on multimodal features fusion (categorized as joint representation in ours).”
Attention, please! A survey of neural attention models in deep learning (2021, background): “1 3 multimodality in neural networks is a historical problem widely discussed by the scientific community (Ramachandram and Taylor 2017; Gao et al. 2020b).”
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks (2022, background): “A domain-and-modality agnostic model that can be adapted to solve tasks across different modalities and domains has become a desidera-tum [14], motivating great efforts in transfer learning [15] and multi-modal learning [16].”
Attention Bottlenecks for Multimodal Fusion (2021, background): “Audiovisual Learning: Audiovisual multimodal learning has a rich history, both before and during the deep learning era [49].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “[10] provide an overview on deep multi-modal learning, and mention its applications in diverse research fields, such as robotic grasping and human action recognition.”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “based on the processing level at which the fusion occurs [158], [209]: early fusion and late fusion (cf.”
Deep Learning for Intelligent Transportation Systems: A Survey of Emerging Trends (2020, neutral): “survey: [79])?”
Deep Learning for Geophysics: Current and Future Trends (2021, background): “In multimodal DL (Ngiam et al., 2011; Ramachandram & Taylor, 2017), inputs are from different sources, such as seismic data and gravity data.”
A Survey of Deep Learning: Platforms, Applications and Emerging Research Trends (2018, background): “Particularly, other works focus primarily on the advances and needs of a single learning mechanism or modality [19], [109], or towards improvements in a single application [38], [67], [68], [98].”
Multimodal deep learning for biomedical data fusion: a review (2022, background): “Thus, fusing modalities gradually rather than all in one layer can be beneficial [2].”
Quantifying and Alleviating the Language Prior Problem in Visual Question Answering (2019, methodology): “The ﬂexibility of deep architecture advances the implementation of multimodal fusion either as feature-level fusion or decision-level fusion [31].”