Incremental semi-supervised learning on streaming data

By Yanchao Li, Yongli Wang, Qi Liu, Cheng Bi, Xiaohui Jiang, Shurong Sun. 2019

58 citations recorded by the source corpus.

Abstract In streaming data classification, most of the existing methods assume that all arrived evolving data are completely labeled. One challenge is that some applications where only small amount of labeled examples are available for training. Incremental semi-supervised learning algorithms have been proposed for regularizing neural networks by incorporating various side information, such as pairwise constraints or user-provided labels. However, it is hard to put them into practice, especially for non-stationary environments due to the effectiveness and parameter sensitivity of such algorithms. In this paper, we propose a novel incremental semi-supervised learning framework on streaming data. Each layer of model is comprised of a generative network, a discriminant structure and the bridge. The generative network uses dynamic feature learning based on autoencoders to learn generative features from streaming data which has been demonstrated its potential in learning latent feature representations. In addition, the discriminant structure regularizes the network construction via building pairwise similarity and dissimilarity constraints. It is also used for facilitating the parameter learning of the generative network. The network and structure are integrated into a joint learning framework and bridged by enforcing the correlation of their parameters, which balances the flexible incorporation of supervision information and numerical tractability for non-stationary environments as well as explores the intrinsic data structure. Moreover, an efficient algorithm is designed to solve the proposed optimization problem and we also give an ensemble method. Particularly, when multiple layers of model are stacked, the performance is significantly boosted. Finally, to validate the effectiveness of the proposed method, extensive experiments are conducted on synthetic and real-life datasets. The experimental results demonstrate that the performance of the proposed algorithms is superior to some state-of-the-art approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Ziwei Liu
5. Jun Zhu
6. Qiwen Wang
7. Liyuan Wang
8. Zhi-Hong Qi

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Ziwei Liu
5. Da-Wei Zhou
6. Liyuan Wang
7. Bernt Schiele
8. Luc Van Gool

In Signal Processing

1. Jun Zhu
2. Chongxuan Li
3. De-chuan Zhan
4. Da-Wei Zhou
5. Ziwei Liu
6. Qiwen Wang
7. Han-Jia Ye
8. Xiatian Zhu

Citations in the corpus

• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning (2021, background): “[17] continually learns partially labeled new instances of fixed classes through stacking generators and discriminators that significantly expand the model.”
• Deep Class-Incremental Learning: A Survey (2023, background): “Since the labeling cost is always expensive in the real world, there are some works addressing training CIL models in a semisupervised [215] or unsupervised manner [171].”
• Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Recently, multiple works explore IL from unlabeled data that comes as a non-stationary stream [283], [291], with the class label space possibly varying over time [292].”