Herding dynamical weights to learn

By Max Welling. 2009

632 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

In Optimization & Training

Citations in the corpus

Learning Energy-Based Spatial-Temporal Generative ConvNets for Dynamic Patterns (2019, methodology): “It can also be considered a generalization of the herding method [59] from exponential family models to general energybased models.”
On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, background): “These works show connections to W-GAN and herding [34].”
Learning Descriptor Networks for 3D Shape Synthesis and Analysis (2018, background): “It is also a generalized version of herding [27] and is related to [1].”
On the Equivalence between Herding and Conditional Gradient Algorithms (2012, neutral): “Welling (2009b) proposed originally an algorithmic interpretation of herding as performing subgradient ascent with constant step size on a function obtained as the zero temperature limit of the log-likelihood of an exponential model that he called the “tipi function”.”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “To alleviate this issue, Bach et al. (2012) shows that the herding procedure of Welling (2009a) takes the form of a convex optimization algorithm in which convergence results can be invoked.”
iCaRL: Incremental Classifier and Representation Learning (2016, background): “The same prioritized construction is used in herding [39] to create a representative set of samples from a distribution.”
Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels (2014, background): “plore two possible approaches for ﬁnding sequences bas ed on optimizing the box discrepancy, namely global optimization and greedy optimization. The latter is closely connected to herding algorithms [Welling, 2009]. Global Adaptive Sequences. The task is posed in terms minimization of the box discre”
Optimally-Weighted Herding is Bayesian Quadrature (2012, methodology): “Herding was introduced by Welling (2009) as a method for generating pseudo-samples from a distribution in such a way that certain nonlinear moments of the sample set closely match those of the target distribution.”
Super-Samples from Kernel Herding (2010, methodology): “We directly describe the herding algorithm in terms of Reproducing Kernel Hilbert Spaces and (potentially) continuous index spaces (note that previous work by [Welling, 2009b,a] cast it in terms of finite-dimensional spaces and discrete domains).”
Statistical inference using weak chaos and infinite memory (2010, background): “(For more details see (Welling, 2009a).)”
FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning (2022, background): “• FeTrIL herd : s features are selected from any new class based on a herding algorithm [43].”
Continual Learning by Modeling Intra-Class Variation (2022, background): “A notable example is iCaRL (Rebuffi et al., 2017), which uses knowledge distillation to transfer the old knowledge in the memory buffer by herding (Welling, 2009) from the previous model to the learned model.”
Continual Learning through Retrieval and Imagination (2022, methodology): “Besides, we consider the adverse effects of an imbalance between new and past tasks, and introduce cosine herding (Welling 2009) sampling strategy, which rebalances new data and past data during each retrieval batch and ensures that all classes are equally represented in the memory.”
Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, methodology): “Specifically, after training on Dt, we apply the herding algorithm (Welling, 2009) to select 20 train-”
Online Continual Learning Via Candidates Voting (2021, background): “• ICARL also adopts distillation loss but it additionally selects fixed number of learned data as exemplars for knowledge replay through Herding algorithm [41].”
Co-Transport for Class-Incremental Learning (2021, background): “The exemplars are selected by herding algorithm [45].”
Rainbow Memory: Continual Learning with a Memory of Diverse Samples (2021, methodology): “These methods include herding selection [45], a discriminative samplings [34] and entropy based samples [6].”
DER: Dynamically Expandable Representation for Class Incremental Learning (2021, methodology): “In these experiemnts, we select exemplars as memory based on the herding selection strategy[32] following the previous works[27].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, methodology): “ADER [148] is proposed to handle CF in session-based recommendation using the adaptive distillation loss and replay with heading [88] technique.”
Adaptive Aggregation Networks for Class-Incremental Learning (2020, methodology): “*: herding is the method to choose exemplars [47], and can be replaced by others, e.”