Layer-wise training of deep networks using kernel similarity

By Mandar Kulkarni, Shirish Subhash Karande. 2017

42 citations recorded by the source corpus.

Deep learning has shown promising results in many machine learning applications. The hierarchical feature representation built by deep networks enable compact and precise encoding of the data. A kernel analysis of the trained deep networks demonstrated that with deeper layers, more simple and more accurate data representations are obtained. In this paper, we propose an approach for layer-wise training of a deep network for the supervised classification task. A transformation matrix of each layer is obtained by solving an optimization aimed at a better representation where a subsequent layer builds its representation on the top of the features produced by a previous layer. We compared the performance of our approach with a DNN trained using back-propagation which has same architecture as ours. Experimental results on the real image datasets demonstrate efficacy of our approach. We also performed kernel analysis of layer representations to validate the claim of better feature encoding.

Subfields: Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Why Should We Add Early Exits to Neural Networks? (2020, methodology): “, through boosting [17, 24], clustering [44], and kernel similarity [33].”
Greedy Layerwise Learning Can Scale to ImageNet (2018, background): “Similarly (Kulkarni & Karande, 2017; Marquez et al., 2018) revisit layerwise training, but in a limited experimental setting.”
Shallow Learning For Deep Networks (2018, background): “Similarly (Kulkarni & Karande, 2017) consider a kernel based layerwise objective, but with unconvincing results in very limited settings.”