A direct adaptive method for faster backpropagation learning: the RPROP algorithm

By Martin A. Riedmiller, H. Braun. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Data-efficient Deep Reinforcement Learning for Dexterous Manipulation (2017, methodology): “with Rprop [28] and using full-batch updates with the entire replay buffer.”
The Graph Neural Network Model (2009, positive): “In our GNN simulator, the weights are updated by the resilient backpropagation [61] strategy, which, according to the literature”
Qsparse-Local-SGD: Distributed SGD With Quantization, Sparsification, and Local Computations (2019, methodology): “Such methods drew interest since RPROP [36], which only used the temporal behavior of the sign of the gradient.”
Deep learning in neural networks: An overview (2014, methodology): “Only the signs of the error derivatives are taken into account by the successful and widely used BP variant R-prop (Riedmiller and Braun, 1993).”
Deep Learning: Methods and Applications (2014, methodology): “Further, effective regularization schedules developed for learning the K-DSN weights can be motivated by intuitive insight from useful optimization tricks such as the heuristic in Rprop or resilient backpropagation algorithm [302].”
signSGD: compressed optimisation for non-convex problems (2018, methodology): “Still Riedmiller & Braun (1993) noted the good practical performance of sign-based methods like RPROP for training deep nets, and since then variants such as RMSPROP (Tieleman & Hinton, 2012) and ADAM (Kingma & Ba, 2015) have become increasingly popular.”
Representing Text for Joint Embedding of Text and Knowledge Bases (2015, neutral): “L-BFGS (Liu and Nocedal, 1989) and RProp (Riedmiller and Braun, 1993) were found to converge to similar function values, with RProp converging signiﬁcantly faster.”
Online Learning Rate Adaptation with Hypergradient Descent (2017, methodology): “es by multiplicatively adapting local learning rates using a meta-learning rate, employing second-order information from fast Hessianvector products [17]. Other work that merits mention include RPROP [20], where local adaptation of weight updates are performed by using only the temporal behavior of ”
Sequential Labeling Using Deep-Structured Conditional Random Fields (2010, methodology): “The RPROP [21] training algorithm was used in all the experiments to learn the model parameters with the initial parameters set to zero and the initial”
Intrusion detection using an ensemble of intelligent paradigms (2005, background): “If the weight continues to change in the same direction for several iterations, then the magnitude of the weight change will be increased (Riedmiller and Braun, 1993).”
Apprenticeship Learning using Inverse Reinforcement Learning and Gradient Methods (2007, methodology): “One significant barrier for applying the method (as well as other methods based on IRL) is that it needs to solve MDPs many times.”
Using a Gaussian mixture neural network for incremental learning and robotics (2012, methodology): “The first row on Table I shows the results obtained using a MLP neural network trained with the resilient propagation (RPROP) learning algorithm [23].”
Batch Reinforcement Learning (2012, methodology): “In particular, within the NFQ framework, the adaptive supervised learning algorithm Rprop [32] is used as the core of the fitting step.”
Automatic Analysis of Facial Expressions: The State of the Art (2000, methodology): “The network has been trained using a resilient propagation [72].”
Bidirectional recurrent neural networks (1997, methodology): “All networks are trained with 200 cycles of a modified version of the resilient propagation (RPROP) technique [10] and extended to a RPROP through a time variant.”
Optimization Algorithms and Applications for Speech and Language Processing (2013, methodology): “Thus, by applying the Taylor series argument mentioned above, we can show that for all sufficiently large—we are guaranteed to see ascent in .”
Reinforcement learning in feedback control (2011, methodology): “In our approach we use Resilient Propagation (RProp) (Riedmiller and Braun 1993) as an epoch-based learning method that proved to be very robust with respect to the choice of the learning parameters and the topology of the network.”