Continual Learning with Bayesian Neural Networks for Non-Stationary Data

By Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick van der Smagt, Stephan Günnemann. 2020

82 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “The second is online variational inference (VI) [55], [56], [57], [58], [59], [60].”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “One of the most popular approaches to relieve forgetting is by imposing a regularization loss within the optimization procedure [7, 11, 13, 16, 19, 25, 26, 31, 34, 35, 40, 41, 57], where the network’s parameters which are important to the past tasks are penalized when updating.”
DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning (2022, methodology): “Regularization-based methods aim to penalize the update either in function space [8, 24] or in parameter space [2, 9, 18, 20, 22, 23, 32, 38, 45].”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…al., 2019; Aljundi, Belilovsky et al., 2019; Aljundi, Lin, Goujaud, & Bengio, 2019; He, Mao, Shao, & Zhu, 2020; Zhao, Xiao, Gan, Zhang, & Xia, 2020; Kurle, Cseke, Klushyn, van der Smagt, & Günnemann, 2020; Chrysakis & Moens, 2020; Kim, Jeong, & Kim, 2020; Tao, Chang, Hong, Wei, & Gong, 2020;…”
Posterior Meta-Replay for Continual Learning (2021, background): “[34] R.”
Meta-Consolidation for Continual Learning (2020, background): “One line of work [88, 38, 47, 2, 16, 71, 56, 13, 79, 42] constrains the parameters of the deep network trained on Task A to not change much while learning a new Task B, while another - replay-based methods - store [63, 48, 14, 66, 9, 3, 4, 22] or generate [74, 81, 46, 45] examples of previous tasks ”
MgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning (2020, neutral): “Recently, there has been a large body of research in incremental learning [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43].”
Task Agnostic Continual Learning via Meta Learning (2019, background): “Furthermore, when additional memory budget are available for storing datapoints, online variational Bayes can also be extended by combining it with memory-based online learning methods (Minka et al., 2009; Nguyen et al., 2017; Kurle et al., 2020).”
VariGrow: Variational Architecture Growing for Task-Agnostic Continual Learning based on Bayesian Novelty (2022, methodology): “Kurle et al. (2019) analyzed non-stationary data using Bayesian neural networks and memory-based online variational Bayes by implementing ‘Bayesian forgetting’ to selectively forget knowledge not relevant to the current data distribution.”
A Unified and General Framework for Continual Learning (2024, unknown): “Representative works include (Nguyen et al., 2018; Kurle et al., 2019; Kao et al., 2021; Henning et al., 2021; Pan et al., 2020; Titsias et al., 2020; Rudner et al., 2022).”
Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping (2021, background): “[24] Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick van der Smagt, and Stephan Günnemann.”