PCANet: A Simple Deep Learning Baseline for Image Classification?

By Tsung-Han Chan, K. Jia, Shenghua Gao, Jiwen Lu, Zinan Zeng, Yi Ma. 2014

1,572 citations recorded by the source corpus.

In this paper, we propose a very simple deep learning network for image classification that is based on very basic data processing components: 1) cascaded principal component analysis (PCA); 2) binary hashing; and 3) blockwise histograms. In the proposed architecture, the PCA is employed to learn multistage filter banks. This is followed by simple binary hashing and block histograms for indexing and pooling. This architecture is thus called the PCA network (PCANet) and can be extremely easily and efficiently designed and learned. For comparison and to provide a better understanding, we also introduce and study two simple variations of PCANet: 1) RandNet and 2) LDANet. They share the same topology as PCANet, but their cascaded filters are either randomly selected or learned from linear discriminant analysis. We have extensively tested these basic networks on many benchmark visual data sets for different tasks, including Labeled Faces in the Wild (LFW) for face verification; the MultiPIE, Extended Yale B, AR, Facial Recognition Technology (FERET) data sets for face recognition; and MNIST for hand-written digit recognition. Surprisingly, for all tasks, such a seemingly naive PCANet model is on par with the state-of-the-art features either prefixed, highly hand-crafted, or carefully learned [by deep neural networks (DNNs)]. Even more surprisingly, the model sets new records for many classification tasks on the Extended Yale B, AR, and FERET data sets and on MNIST variations. Additional experiments on other public data sets also demonstrate the potential of PCANet to serve as a simple but highly competitive baseline for texture classification and object recognition.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification (2018, methodology): “Six state-of-the-art methods are reported to have achieved promising results on the aforementioned benchmark datasets in the literature [3].”
TI-POOLING: Transformation-Invariant Pooling for Feature Learning in Convolutional Neural Networks (2016, neutral): “48 PCANet-2 [5] 7.”
An Interpretable Generative Model for Handwritten Digits Synthesis (2019, result): “It is worthwhile to point out that our work is completely different from the PCA-net in [31].”
Optimizing the Latent Space of Generative Networks (2017, methodology): “For instance, PCA has been used to learn low-dimensional representations of human faces [Turk and Pentland, 1991], or to produce a hierarchy of features [Chan et al., 2015].”
Semantic Edge Detection with Diverse Deep Supervision (2018, background): “The unifying idea is deep learning which utilizes neural networks with many hidden layers aimed at learning complex feature representations from raw data (Chan et al. 2015; Liu et al. 2018; Tang et al. 2017).”
Building Efficient Deep Hebbian Networks for Image Classification Tasks (2017, methodology): “Comparison to other models: On the MIT-67, the DHN shows higher accuracy than the PCANet [5], Deformable Parts models (DPM), Spatial Pyramid Matching (SPM) [13] and Reconfigurable Models (RBoW) [14], as reported in Table 2.”
Generalized BackPropagation, Étude De Cas: Orthogonality (2016, methodology): “Chan et al. propose PCANets, a hierarchical structure with orthogonal filter-banks to address the problem of image classification Chan et al. (2015). The orthogonal filter-banks are learned by applying PCA over blocks extracted in the vicinity of the filters.”
On Completeness-aware Concept-Based Explanations in Deep Neural Networks (2019, positive): “Some use dimensionality reduction methods [34, 35], while others uncover higher level human-relatable concepts by dimensionality reduction (e.g. for speech [36] and language [37, 38]).”
Local binary features for texture classification: Taxonomy and experimental study (2017, positive): “In summary, we can observe a general trend of performance increase with neighborhood size, with most LBP methods achieving a best performance beyond three scales, clearly indicating the necessity of using larger areas of spatial support for LBP feature extraction.”
VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition (2017, methodology): “Bailo et al. [2] proposes a method that extracts multiple regions of interest as MSERs [25], merges regions that possibly belong to the same class, and finally classifies region proposals by utilizing a PCANet [6] and a neural network.”