An Equivalence of Fully Connected Layer and Convolutional Layer

By Wei Ma, Jun Lu. 2017

79 citations recorded by the source corpus.

This article demonstrates that convolutional operation can be converted to matrix multiplication, which has the same calculation way with fully connected layer. The article is helpful for the beginners of the neural network to understand how fully connected layer and the convolutional layer work in the backend. To be concise and to make the article more readable, we only consider the linear case. It can be extended to the non-linear case easily through plugging in a non-linear encapsulation to the values like this $\sigma(x)$ denoted as $x^{\prime}$.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Wang
2. Wei Pan
3. Minghao Yang
4. Hongpeng Zhou
5. Shiqing Ma
6. X. Zhang
7. Yingqi Liu
8. Wen-Chuan Lee

In Statistical and Nonlinear Physics

1. X. Zhang
2. Wei Pan
3. Minghao Yang
4. Jun Wang
5. Hongpeng Zhou

Citations in the corpus

• BayesNAS: A Bayesian Approach for Neural Architecture Search (2019, methodology): “As in (Ma & Lu, 2017), Bi is converted to two dimensional matrix for FC layer, with dimension (bHojW o j )× (Cimoijkoij).”
• ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation (2019, background): “According to [22, 41], convolutional layers can be converted to equivalent fully connected layers.”