Understanding convolutional neural networks with a mathematical model

By C.-C. Jay Kuo. Journal of Visual Communication and Image Representation, 2016

405 citations recorded by the source corpus.

Abstract This work attempts to address two fundamental questions about the structure of the convolutional neural networks (CNN): (1) why a nonlinear activation function is essential at the filter output of all intermediate layers? (2) what is the advantage of the two-layer cascade system over the one-layer system? A mathematical model called the “REctified-COrrelations on a Sphere” (RECOS) is proposed to answer these two questions. After the CNN training process, the converged filter weights define a set of anchor vectors in the RECOS model. Anchor vectors represent the frequently occurring patterns (or the spectral components). The necessity of rectification is explained using the RECOS model. Then, the behavior of a two-layer RECOS system is analyzed and compared with its one-layer counterpart. The LeNet-5 and the MNIST dataset are used to illustrate discussion points. Finally, the RECOS model is generalized to a multilayer system with the AlexNet as an example.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/52d7ae292f285ab24b050b8d229ac98cd674523c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiali Duan
2. Yueru Chen
3. Siyang Li
4. Min Zhang
5. Lihua Xie
6. Chen Wang
7. Jianfei Yang
8. Junsong Yuan

Citations in the corpus

• Kervolutional Neural Networks (2019, background): “While this is not optimal, especially when considering that the mechanism of deep architectures is still unclear [32].”
• Interpretable Convolutional Neural Networks via Feedforward Design (2018, methodology): “Efforts have been made to explain the roles played by anchor vector a k and nonlinear activation φ p¨q in [20, 21, 22].”
• An Interpretable Generative Model for Handwritten Digits Synthesis (2019, background): “and his colleagues have published a sequence of papers on interpretable convolutional neural networks (CNNs) [26, 27, 28, 29, 30].”