Revisiting multiple instance neural networks

By Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai. Pattern Recognition, 2016

517 citations recorded by the source corpus.

We revisit the problem of solving MIL using neural networks (MINNs), which are ignored in current MIL research community. Our experiments show that MINNs are very effective and efficient.We proposed a novel MI-Net which is centered on learning bag representation in the neural networks in an end-to-end way.Recent deep learning tricks including dropout, deep supervision and residual connections are studied in MINNs. We find deep supervision and residual connections are effective for MIL.In the experiments, the proposed MINNs achieve state-of-the-art or competitive performance on several MIL benchmarks. Moreover, it is extremely fast for both testing and training, for example, it takes only 0.0003s to predict a bag and a few seconds to train on MIL datasets on a moderate CPU. Of late, neural networks and Multiple Instance Learning (MIL) are both attractive topics in the research areas related to Artificial Intelligence. Deep neural networks have achieved great successes in supervised learning problems, and MIL as a typical weakly-supervised learning method is effective for many applications in computer vision, biometrics, natural language processing, and so on. In this article, we revisit Multiple Instance Neural Networks (MINNs) that the neural networks aim at solving the MIL problems. The MINNs perform MIL in an end-to-end manner, which take bags with a various number of instances as input and directly output the labels of bags. All of the parameters in a MINN can be optimized via back-propagation. Besides revisiting the old MINNs, we propose a new type of MINN to learn bag representations, which is different from the existing MINNs that focus on estimating instance label. In addition, recent tricks developed in deep learning have been studied in MINNs; we find deep supervision is effective for learning better bag representations. In the experiments, the proposed MINNs achieve state-of-the-art or competitive performance on several MIL benchmarks. Moreover, it is extremely fast for both testing and training, for example, it takes only 0.0003 s to predict a bag and a few seconds to train on MIL datasets on a moderate CPU.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0820ccfdba775c304bedb9c3d82ee8758e0a416b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Hopfield Networks is All You Need (2020, background): “The Hopfield layer in a deep neural network architecture can implement: • a memory (storage) with associative retrieval [20, 5], • conditional pooling and averaging operations [67, 39], • combining data by associations [2], • associative credit assignment (e.”
Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology (2018, background): “Comparison studies have found little or no improvement from these MI methods on some data sets [14,15].”
Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network (2017, methodology): “To incorporate the MIL into deep neural network framework, the key component is a layer that combines the information from different instances together, which is called MIL Pooling Layer (MPL [26]).”
Attention-based Deep Multiple Instance Learning (2018, unknown): “It is advocated in (Wang et al., 2016) that the latter approach is preferable in terms of the bag level classification performance.”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “Early methods in this ﬁled usually add prior knowledge [137] or post regularization [6] on the classic MIL models, such as LSVM [185], while the current research obtains the break-through by building deep MIL models [168], [175], [216].”
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection (2018, background): “Many variants have been proposed for MIL [14], [23], [24], [25].”
TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication (2021, methodology): “In deep learning based MIL, one straightforward idea is to perform pooling operation [6, 7] on instance feature embeddings extracted by CNN.”
Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning (2020, methodology): “The embedding-based method produces a bag score based on a bag embedding directly supervised by the bag label and usually yields better accuracy compared to the instance-based method [50], however, it is usually harder to determine the key instances that trigger the classifier[26].”
Modern Hopfield Networks and Attention for Immune Repertoire Classification (2020, methodology): “The second approach is to construct an instance representation zi of each instance by and then encode the bag, or the input object, by pooling these instance representations (Wang et al., 2018) via a function f.”
DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification (2022, methodology): “MIL models generally can be divided into two groups, based on whether the final bag predictions are directly from instance predictions [3, 9, 12, 15, 17, 47], or from the aggregations of features of instance [14, 18, 23, 34, 35, 42, 53].”