Online Deep Learning: Learning Deep Neural Networks on the Fly

By Doyen Sahoo, Quang Pham, Jing Lu, Steven C. H. Hoi. 2017

363 citations recorded by the source corpus.

Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch setting, requiring the entire training data to be made available prior to the learning task. This is not scalable for many real-world scenarios where new data arrives sequentially in a stream. We aim to address an open challenge of ``Online Deep Learning" (ODL) for learning DNNs on the fly in an online setting. Unlike traditional online learning that often optimizes some convex objective function with respect to a shallow model (e.g., a linear/kernel-based hypothesis), ODL is more challenging as the optimization objective is non-convex, and regular DNN with standard backpropagation does not work well in practice for online settings. We present a new ODL framework that attempts to tackle the challenges by learning DNN models which dynamically adapt depth from a sequence of training data in an online learning setting. Specifically, we propose a novel Hedge Backpropagation (HBP) method for online updating the parameters of DNN effectively, and validate the efficacy on large data sets (both stationary and concept drifting scenarios).

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares (2022, critical): “Further, [15] proposed a one-pass deep learning algorithm, but it relies on a specific network architecture and is vulnerable to forgetting the previous data unless the data is consistently arriving from the same distribution.”
Task-Free Continual Learning (2018, neutral): “Some recent works include [32, 27], who both start from a small network and then adapt the capacity by adding more neurons as new samples arrive, while for online deep metric learning, [18] proposed a method based on stacking multiple metric layers.”
Continual Few-Shot Learning for Text Classification (2021, background): “Online learning algorithms learn to update models from data streams sequentially, where the task is the same but can exhibit concept drift (new patterns) (Zinkevich, 2003; Crammer et al., 2006; Sahoo et al., 2018; Jerfel et al., 2019; Javed and White, 2019).”
Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL (2018, background): “In practice, their performance with deep neural network function ap-proximators is limited (Sahoo et al., 2017): such high-dimensional models must be trained with batch-mode methods, minibatches, and multiple passes over the data.”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, background): “It is thus not surprising that with the advent of DL immediate attempts have been made to consider online learning in DNNs (Zhou et al., 2012), see a recent survey by Sahoo et al. (2018).”
Continual Learning in Neural Networks (2019, background): “Some recent works include [135, 119], start from a small network and then adapt the capacity by adding more neurons as new samples arrive.”
Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability (2019, methodology): “Considering IADM is related to the deep incremental learning with concept drift, several state-of-the-art methods are compared, i.e., DNN-SGD, Adwin [3], ODL [31].”
Online Learning: A Comprehensive Survey (2018, background): “Fifth, due to the remarkable successes and impact of deep learning techniques for various applications in recent years, another emerging and increasingly important topic is “online deep learning” (Sahoo et al., 2018), i.”
Deep Neural Networks and Tabular Data: A Survey (2021, methodology): “Recent work shows that some of the limitations of neural networks in an evolving data stream can be overcome [25], [212].”
A CCELERATING O NLINE R EINFORCEMENT L EARNING VIA M ODEL -B ASED M ETA -L EARNING (2021, background): “While gradient-based SGD can be used for online learning (Bottou, 1998), its performance is limited with deep neural networks (Sahoo et al., 2018), which work best when trained over large batches of data.”
Cost-Effective Incremental Deep Model: Matching Model Capacity With the Least Sampling (2023, neutral): “However, in the incremental setting, different depths are suitable for different numbers of instances [27], i.”
Deep Online Learning via Meta-Learning : Continual Adaptation for Model-Based RL (2019, background): “…the oldest applications of SGD (Bottou, 1998), modern parametric models such as deep neural networks are difficult to train online with this method (Sahoo et al., 2017) because they require medium-sized minibatches and multiple epochs to arrive at sensible solutions, which is not suitable for…”
Contextual Transformation Networks for Online Continual Learning (2021, background): “This problem is important for deep neural networks because optimizing them in the online setting has been shown to be challenging (Sahoo et al., 2018; Aljundi et al., 2019a).”
Continual Learning, Fast and Slow (2021, background): “On the other hand, although deep neural networks have achieved impressive results in many applications [9], they often require having access to a large amount of data while performing poorly when learning on data streams [10], [11].”
Continual Backprop: Stochastic Gradient Descent with Persistent Randomness (2021, background): “However, a primary challenge to leveraging the strengths of deep learning beyond current applications is that Backprop does not work well in non-stationary problems (McCloskey and Cohen 1989, French 1997, Sahoo et al. 2018), for example, a problem that consists a sequence of stationary problems.”
Machine Learning and Integrative Analysis of Biomedical Big Data (2019, background): “These deep learning approaches are memory efficient as entire datasets need not be stored in memory, making them attractive for large-scale multi-omics analysis [206,207].”
A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications (2023, background): “Stochastic or online gradient descent that suits ANN is an online learning model [119] that lowers generalization error during online training, in which mini lots or instances are derived from dataset [119, 122].”