Deep Bayesian Unsupervised Lifelong Learning
By Tingting Zhao, Zifeng Wang, Aria Masoomi, Jennifer G. Dy. 2021
40 citations recorded by the source corpus.
Lifelong Learning (LL) refers to the ability to continually learn and solve new problems with incremental available information over time while retaining previous knowledge. Much attention has been given lately to Supervised Lifelong Learning (SLL) with a stream of labelled data. In contrast, we focus on resolving challenges in Unsupervised Lifelong Learning (ULL) with streaming unlabelled data when the data distribution and the unknown class labels evolve over time. Bayesian framework is natural to incorporate past knowledge and sequentially update the belief with new data. We develop a fully Bayesian inference framework for ULL with a novel end-to-end Deep Bayesian Unsupervised Lifelong Learning (DBULL) algorithm, which can progressively discover new clusters without forgetting the past with unlabelled data while learning latent representations. To efficiently maintain past knowledge, we develop a novel knowledge preservation mechanism via sufficient statistics of the latent representation for raw data. To detect the potential new clusters on the fly, we develop an automatic cluster discovery and redundancy removal strategy in our inference inspired by Nonparametric Bayesian statistics techniques. We demonstrate the effectiveness of our approach using image and text corpora benchmark datasets in both LL and batch settings.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Oncology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zizhao Zhang
- Tomas Pfister
- Jennifer G. Dy
- Ruoxi Sun
- Vincent Perot
- Han Zhang
- Zifeng Wang
- Sayna Ebrahimi
In Computer Vision and Pattern Recognition
- Zizhao Zhang
- Luc Van Gool
- Tomas Pfister
- Han Zhang
- Trevor Darrell
- Dacheng Tao
- Sayna Ebrahimi
- Ming-Hsuan Yang
Citations in the corpus
- DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, methodology): “These methods can be further categorized as expanding the model [51, 65, 26, 31, 59], or dividing the model [35, 52, 58, 17, 11].”
- Learning to Prompt for Continual Learning (2021, background): “The task-specific components can be identified by expanding the network [26, 31, 48, 50, 64, 68], or attend to task-specific sub-networks [19,35,51,59].”