On Line Learning In Neural Networks

By H. Freud. 2016

145 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Model Architectures, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Model Architectures

In Optimization & Training

Citations in the corpus

Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning (2018, background): “When peers draw a sample from this data to compute an SGD update, we assume that this is done uniformly and independently [13].”
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “…series of classic papers by Biehl & Schwarze (1995) and Saad & Solla (1995a) who derived a set of closed ODEs that track the test error of the student (see also Saad & Solla (1995b); Biehl et al. (1996); Saad (2009) for further results and Goldt et al. (2019) for a recent proof of these equations).”
Assumed Density Filtering Methods for Learning Bayesian Neural Networks (2016, background): “…2014; Hern´andez-Lobato and Adams 2015), where the authors build upon the online Bayesian inference algorithm known as assumed density ﬁl-tering (ADF) (Opper 1998) to develop methods for learning Bayesian neural networks — expectation backpropagation (EBP) and probabilistic backpropagation (PBP).”
Understanding Short-Horizon Bias in Stochastic Meta-Optimization (2018, background): “A variety of decay schedules have been proposed for different architectures, including polynomial, exponential, staircase, etc. Learning rate decay is also required to achieve convergence guarantee for stochastic gradient methods under certain conditions (Bottou, 1998).”
Metaheuristic design of feedforward neural networks: A review of two decades of research (2017, background): “, data comes in sequential order, and traditionally, re-training based mechanics for dynamic learning (online learning) of FNN is the basic option [342].”
A Berkeley View of Systems Challenges for AI (2017, background): “Some aspects of learning in dynamic environments are addressed by online learning [17], in which data arrive temporally and updates to the model can occur as new data arrive.”
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2020, unknown): “In this case, SGD is suitable for online learning [77].”
Star-galaxy Classification Using Deep Convolutional Neural Networks (2016, methodology): “We then use SExtractor to ﬁnd the pixel positions of the 65,000 objects we have selected, and to center each object in the cutout image.”
Gradient Descent Learns Linear Dynamical Systems (2016, methodology): “We defer the proof of Proposition 2.4 to Appendix A which is a simple variation of the standard convergence analysis of stochastic gradient descent (see, for example, [Bot98]).”