Structural learning with forgetting

By Masumi Ishikawa. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

BayesNAS: A Bayesian Approach for Neural Architecture Search (2019, background): “The conventional approach to achieve sparsity is by enforcing penalty terms (Chauvin, 1989; Weigend et al., 1991; Ishikawa, 1996).”
Sparse Networks from Scratch: Faster Training without Losing Performance (2019, methodology): “Earlier work focused on pruning via second-order derivatives (LeCun et al., 1989; Karnin, 1990; Hassibi and Stork, 1992) and heuristics which ensure efﬁcient training of networks after pruning (Chauvin, 1988; Mozer and Smolensky, 1988; Ishikawa, 1996).”
SNIP: Single-shot Network Pruning based on Connection Sensitivity (2018, methodology): “The methods from the former category (Chauvin (1989); Weigend et al. (1991); Ishikawa (1996)) augment the loss function with some sparsity enforcing penalty terms ( e.g ., L 0 or L 1 norm), so that back-propagation effectively penalizes the magnitude of the weights during training.”
Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning (2001, methodology): “These are methods and systems for: adaptive learning [4,5,7,8,14,30,46,47,48]; incremental learning [6,7,8,9,19,53,58,61,71]; lifelong learning [69,35,36,82]; on-line learning [17,21,22,28,31,35,36,42,44,61,66,67,69]; constructivist structural learning [15,19,11,14,9] that is supported by biological”
DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction (2002, unknown): “At the same time, several models that have elements of adaptive, online learning or structure and knowledge adaptation, have been developed that include connectionist models [1]–[4], [10]–[12], [16], [19], [21], [23], [25], [26], [30], [31], [33], [34], [45]–[49], [54], [57], [61], [64], [65], fuzzy”
Extracting rules from trained neural networks (2000, background): “Although several algorithms have been proposed by several researchers [4], [7], there is no algorithm which can be applied to any type of networks, to any training algorithm, and to both discrete and continuous values.”