Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition

By Kunihiko Fukushima, S. Miyake. 1982

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “The promise of unsupervised representation learning has existed since the earliest days of learning in computer vision [20], inspired by the fact that humans and animals need little supervision to perform a range of complex visual tasks.”
Making Convolutional Networks Shift-Invariant Again (2019, background): “Local connectivity and weight sharing have been a central tenet of neural networks, including the Neocognitron (Fukushima & Miyake, 1982), LeNet (LeCun et al., 1998) and modern networks such as Alexnet (Krizhevsky et al., 2012), VGG (Simonyan & Zisserman, 2015), ResNet (He et al., 2016), and DenseNe”
Why do deep convolutional networks generalize so poorly to small image transformations? (2018, background): “In particular, the choice of convolution and pooling in CNNs is motivated by the desire to endow the networks with invariance to irrelevant cues such as image translations, scalings, and other small deformations (Fukushima and Miyake, 1982; Zeiler and Fergus, 2014).”
Recent advances in convolutional neural networks (2015, methodology): “Inspired by this discovery, Kunihiko Fukushima proposed the neocognitron in 1980 [2], which could be regarded as the predecessor of CNN.”
Recognition in Terra Incognita (2018, background): “Thanks to the combination of deep learning [6, 7], Moore’s law [8] and very large annotated datasets [9, 10] enormous progress has been made during the past 10 years.”
Multiscale Vision Transformers (2021, background): “Fukushima proposed the Neocognitron [37], a neural network architecture for pattern recognition explicitly motivated by Hubel and Wiesel’s hierarchy.”
Improved multitask learning through synaptic intelligence (2017, background): “Although originally inspired by biology (Fukushima & Miyake, 1982), the underlying design principles and learning methods differ substantially from biological neural networks.”
The Devil is in the Tails: Fine-grained Classification in the Wild (2017, background): “…such progress: (a) the invention of ‘deep network’ algorithms where visual computation is learned from the data rather than hand-crafted by experts [14, 27, 29], (b) the design and construction of large and well annotated datasets [8, 11, 13, 30] supplying researchers with a sufﬁcient amount of…”
A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information (1993, background): “Note November 1993, 13(11) 4715 that such information is typically lost in networks that utilize feature hierarchies of complex cells (Fukushima, 1980, 1987; LeCun et al., 1990) or Fourier transforms (e.g., Pollen et al., 1971; Cavanagh, 1978, 1985) for forming position, scale, and or…”
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (2018, background): “…well suited to modeling the world include: hierarchy and compositionality (Lin et al., 2017; Poggio et al., 2017), Markovian dynamics (Tiňo et al., 2004; 2007), and equivariances in time and space for RNNs (Werbos, 1988) and CNNs (Fukushima & Miyake, 1982; Rumelhart et al., 1985) respectively.”
A survey of the recent architectures of deep convolutional neural networks (2019, background): “In 1989, LeCuN et al. proposed the first multilayered CNN named ConvNet, whose origin rooted in Fukushima’s Neocognitron (Fukushima and Miyake 1982; Fukushima 1988).”
Deep Learning on Image Denoising: An overview (2019, unknown): “The original deep learning technologies were found in image processing in 1980s [53] and were used in image denoising in 1980s by Zhou et al.”
Deep Learning Applications in Medical Image Analysis (2018, background): “CNNs may have their origins in the Neocognitron concept proposed by Fukushima [8] in 1982, but it was Lecun et al. [9] who formalized CNNs and used the error backpropagation described by Rumelhart et al. [10], to successfully perform the automatic recognition of hand-written digits.”
Hyena Hierarchy: Towards Larger Convolutional Language Models (2023, background): “The classical approach of convolutional neural networks (CNNs) (Fukushima and Miyake, 1982; LeCun et al., 1998; Ronneberger et al., 2015; He et al., 2016) is to optimize directly the values ht of the filter’s response at M prescribed steps, a parametrization we call explicit.”
Iris Recognition With Off-the-Shelf CNN Features: A Deep Learning Perspective (2018, background): “Later, Fukushima and Miyake [31] proposed the NeoCognitron, which could be regarded as the predecessor of the CNN.”
Absence of Barren Plateaus in Quantum Convolutional Neural Networks (2020, methodology): “The architecture of CNNs was inspired by seminal experiments investigating the structure of the visual cortex [3, 4].”