Cognitron: A self-organizing multilayered neural network

By Kunihiko Fukushima. 1975

Subfields: Model Architectures, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Neuroscience

Citations in the corpus

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, background): “Its complexity is still quadratic to image size, while ours is linear and also operates locally which has proven beneficial in modeling the high correlation in visual signals [32, 22, 37].”
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (2018, background): “Locally Connected Networks (LCNs) (Fukushima, 1975; Lecun, 1989) are CNNs without weight sharing between spatial locations.”
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? (2023, background): “In this work, we ask the same question that Fukushima [1, 2] asked nearly 50 years ago – how do we design an artificial visual cortex , the module in a computational system that (together with a policy) enables an agent to convert camera input into actions?”
Deep learning in bioinformatics (2016, methodology): “However in this review, we use the term “DNN” to specifically refer to multilayer perceptron (MLP) [104], stacked auto-encoder (SAE) [105, 106], and deep belief network (DBN) [107, 108] which uses perceptron [120], auto-encoder (AE) [121], and restricted Boltzmann machine (RBM) [122, 123] as a build”
Sparse Autoencoders Find Highly Interpretable Features in Language Models (2023, methodology): “We use the ReLU activation function in the hidden layer (Fukushima, 1975).”
Understanding of Object Detection Based on CNN Family and YOLO (2018, background): “From the early concept of the receptive fields [2] and the improved Neural Network (NN) models (Rosenblatt [3] in 1962; Kabrisky [4] in 1966; Giebel [5] in 1971; Fukushima [6] in 1975) to Back Propagation Neural Netowrk (BP_NN) [7, 8] in 1986.”
Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics (2021, background): “…of MLPs, recurrent neural networks (RNN), LSTMs (Hochre-iter & Schmidhuber, 1997), gated recurrent unit (GRU) (Cho et al., 2014), convolutions (Fukushima, 1980; 1975; Lecun et al., 1998; 2000; Rumelhart et al., 1986), residual connections (He et al., 2016; Huang et al., 2017), batch…”
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes (2019, background): “(GRU) [10]), skip connections [24, 27], convolutions [16, 17, 47, 35, 36] or graph convolutions [8, 25, 15, 38, 14, 31], pooling [35, 36], batch normalization (batchnorm) [28], layer normalization [1] and/or attention [2, 55].”
Convolutional face finder: a neural architecture for fast and robust face detection (2004, unknown): “The idea of connecting units to local receptive fields on the input was largely inspired by Hubel and Wiesel’s discovery of locally-sensitive, orientationselective neurons in the cat visual system [34] and local connections have been used many times in neural models of visual learning [28], [35], [3”
A Survey of Deep Learning Methods for Cyber Security (2019, background): “Although ReLUs were originally proposed in the 1970s [17], they gained widespread use”
Locality defeats the curse of dimensionality in convolutional teacher–student scenarios (2021, background): “Such functions model the local nature of certain datasets and can be generated, for example, by randomly-initialised one-hidden-layer neural networks: f LC corresponds to a locally connected network (LCN) [10, 11], in which the input is split into lower-dimensional patches before being processed, wh”
Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data (2019, unknown): “, 2012) (E) AE (Fukushima, 1975; Krizhevsky and Hinton, 2011) (F) Sparse AE (Vincent et al.”
Tensor Programs II: Neural Tangent Kernel for Any Architecture (2020, background): “, Long-Short Term Memory (LSTM) [33] or Gated Recurrent Unit (GRU) [14]), skip connections [31, 35], convolutions [22, 23, 42, 43, 57] or graph convolutions [11, 17, 20, 32, 40, 46], pooling [42, 43], batch normalization [38], layer normalization [8] and/or attention [9, 59].”