Attention-based Deep Multiple Instance Learning
By Maximilian Ilse, J. Tomczak, Max Welling. 2018
2,646 citations recorded by the source corpus.
Multiple instance learning (MIL) is a variation of supervised learning where a single class label is assigned to a bag of instances. In this paper, we state the MIL problem as learning the Bernoulli distribution of the bag label where the bag label probability is fully parameterized by neural networks. Furthermore, we propose a neural network-based permutation-invariant aggregation operator that corresponds to the attention mechanism. Notably, an application of the proposed attention-based operator provides insight into the contribution of each instance to the bag label. We show empirically that our approach achieves comparable performance to the best MIL methods on benchmark MIL datasets and it outperforms other methods on a MNIST-based MIL dataset and two real-life histopathology datasets without sacrificing interpretability.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Medical Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Jianfeng Gao
- Mohit Bansal
- Quoc V. Le
- Dacheng Tao
- Sergey Levine
- Ruslan Salakhutdinov
- Percy Liang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Yi Yang
- Andrea Vedaldi
- Trevor Darrell
- Jitendra Malik
- Xiaogang Wang
In Medical Imaging
- Fanjie Kong
- Ricardo Henao
- Faisal Mahmood
- Tiffany Y. Chen
- Ming Y. Lu
- Drew F. K. Williamson
- Richard J. Chen
- Anne L. Martel
Citations in the corpus
- End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, neutral): “On the other hand, the attention-based approach [36] (Attn+NCE) computes cross-modal attention weights between all the clipcaption pairs and perform a weighted average of the similarity scores in order to consider the most relevant positive candidate pairs.”
- Set Transformer (2018, positive): “• rFF + Dotprod (Yang et al., 2018; Ilse et al., 2018): rFF layers in encoder and dot product attention based weighted sum pooling + rFF layers in decoder.”
- Set2Graph: Learning Graphs From Sets (2020, methodology): “Instead of using a max or sum pooling in DeepSets layers, we used a self-attention mechanism based on (Ilse et al., 2018) and (Vaswani et al., 2017): We used a grid search for the following hyper-parameters: learning rate in the range of { 1 e − 5 , 3 e − 5 , 1 e − 4 , 3 e − 4 , 1 e − 3 , 3 e − 3 }…”
- Spotting Insects from Satellites: Modeling the Presence of Culicoides Imicola Through Deep CNNs (2019, methodology): “We inherit from [23] a general three-steps approach for classifying a bag of instances: i) to apply a non-linear transformation to instances trough a function f ; ii) to exploit a symmetric”
- Differentiable Patch Selection for Image Recognition (2021, background): “Multiple-Instance Learning : A number of works use attention to solve Multiple-Instance Learning (MIL) problems, which are especially common in medical imaging, where images tend to be very large [20, 27].”
- Efficient Classification of Very Large Images with Tiny Objects (2021, methodology): “Following [16,20], we treat the problem as a binary classification task where the positive images are those containing at least one cell belonging to the epithelial cell class.”
- Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning (2020, methodology): “Recent MIL based approaches have greatly benefited from using deep neural networks for feature extraction and feature aggregation [21, 47, 33].”
- Deep Learning for Whole Slide Image Analysis: An Overview (2019, background): “One of the most promising emerging directions aims at incorporating the process of selecting patches within the optimisation of visual understanding [18, 25, 39, 6, 35, 45].”
- Deep learning in cancer diagnosis, prognosis and treatment selection (2021, methodology): “the model employed an explainability method called attention [109, 110] to assign diagnostic relevance scores to image regions and revealed that regions”
- A foundation model for clinical-grade computational pathology and rare cancers detection (2024, positive): “Using multiple-instance learning 14,49,55 with labels at the level of groups of slides has enabled clinically relevant diagnostics by scaling to training datasets on the order of 10,000 WSIs 10–14 .”
- TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication (2021, methodology): “Traditional MIL methods as well as the latest MIL methods such as ABMIL, DSMIL and CLAM usually require a large number of epochs to converge.”
- Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction (2018, background): “However, [23] aims to solve the long-term memory problem for sequence learning, while [12] focuses on multiple instance learning for classification.”
- Modern Hopfield Networks and Attention for Immune Repertoire Classification (2020, methodology): “A model ŷ = g(X) should be (a) invariant to permutations of the instances and (b) able to cope with the fact that N varies across input objects (Ilse et al., 2018), which is a problem also posed by point sets (Qi et al., 2017).”