Deep Compositional Question Answering with Neural Module Networks
By Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Daniel Klein. 2015
167 citations recorded by the source corpus.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Dacheng Tao
- Trevor Darrell
- Zhe Gan
- Yoshua Bengio
- Chunhua Shen
- Mohit Bansal
- Abhinav Kumar Gupta
- Luc Van Gool
Citations in the corpus
- Learning modular neural network policies for multi-task and multi-robot transfer (2016, neutral): “Domain adaptation techniques have been used to adapt training data in the face of systematic domain shift [23], and more recently, work on modular networks for visual question answering has been demonstrated with good results [24].”
- Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering (2017, methodology): “Neural Module Networks (NMN) [3]: The model designed to be compositional in nature.”
- An Analysis of Visual Question Answering Algorithms (2017, methodology): “DAQUAR and COCO-QA are small and have a limited variety of question-types.”
- Routing Networks and the Challenges of Modular and Compositional Computation (2019, background): “To address this issue, a number of models have been introduced to enforce specialization [Jacobs et al., 1991b, Miikkulainen, 1993, Jordan and Jacobs, 1994, Bengio et al., 2013, Davis and Arel, 2013, Andreas et al., 2015, Bengio et al., 2015, Shazeer et al., 2017, Fernando et al., 2017].”
- Reasoning about Pragmatics with Neural Listeners and Speakers (2016, background): “Reasoning with neural networks A general framework for constructing task-specific neural networks and composing them to produce novel behavior was explored in the context of visual question answering by Andreas et al. (2015).”
- Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning (2018, background): “While conceptually similar to prior work in modular networks [2, 10, 14], our model differs drastically in module design.”
- Toward an Integration of Deep Learning and Neuroscience (2016, background): “For example, sub-dividing networks into units that can be modularly and dynamically combined, where representations can be copied and routed, may present a path toward improved compositionality and transfer learning (Andreas et al., 2015).”
- Answer Them All! Toward Universal Visual Question Answering Models (2019, methodology): “While CVQA and VQACPv2 try to overcome these issues, synthetic datasets [19, 6, 21] minimize such biases to a greater extent, and serve as an important litmus-test to measure specific reasoning skills, but the synthetic visual scenes lack complexity and variation.”
- Image Captioning and Visual Question Answering Based on Attributes and External Knowledge (2016, methodology): “Inspired by Xu et al. [39] who encode visual attention in the Image Captioning, [46], [47], [48], [49], [50], [51] propose to use the spatial attention to help answering visual questions.”
- Deep Visual-Semantic Hashing for Cross-Modal Retrieval (2016, background): “To our knowledge, this work is the first end-to-end learning approach to cross-modal deep hashing that enables efficient cross-modal retrieval of images in response to text-sentences queries and vice versa.”
- Training Recurrent Answering Units with Joint Loss Minimization for VQA (2016, background): “[15, 16] propose neural module networks, which combines multiple module networks to construct a single deep network.”
- Multimodal Intelligence: Representation Learning, Information Fusion, and Applications (2019, background): “neural “modules” combined into a deep model for answering questions [259].”
- Data Augmentation for Visual Question Answering (2017, methodology): “Some notable formulations include attention based methods (Yang et al., 2016; Xiong et al., 2016; Lu et al., 2016; Fukui et al., 2016), Bayesian frameworks (Kafle and Kanan, 2016; Malinowski and Fritz, 2014), and compositional approaches (Andreas et al., 2016a,b).”
- Synthetic Data for Deep Learning (2019, background): “This line of work is based on neural module networks [21, 91] and similar constructions [22, 259], where the network learns to create a composition of modules (in QA, based on parsing the question) that are also neural networks, all learned jointly (see, however, the critique in [533]).”
- Hierarchical Question-Image Co-Attention for Visual Question Answering (2016, background): “[1] propose a compositional scheme that consists of a language parser and a number of neural modules networks.”