Deep Clustered Convolutional Kernels

By Minyoung Kim, Luca Rigazio. 2015

18 citations recorded by the source corpus.

Deep neural networks have recently achieved state of the art performance thanks to new training algorithms for rapid parameter estimation and new regularization methods to reduce overfitting. However, in practice the network architecture has to be manually set by domain experts, generally by a costly trial and error procedure, which often accounts for a large portion of the final system performance. We view this as a limitation and propose a novel training algorithm that automatically optimizes network architecture, by progressively increasing model complexity and then eliminating model redundancy by selectively removing parameters at training time. For convolutional neural networks, our method relies on iterative split/merge clustering of convolutional kernels interleaved by stochastic gradient descent. We present a training algorithm and experimental results on three different vision tasks, showing improved performance compared to similarly sized hand-crafted architectures.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Frank Hutter
  3. Barret Zoph
  4. Esteban Real
  5. Alan L. Yuille
  6. Xuanyi Dong
  7. Xia Hu
  8. Xiaowen Chu

In Computer Vision and Pattern Recognition

  1. Quoc V. Le
  2. Dacheng Tao
  3. Alan L. Yuille
  4. Luc Van Gool
  5. Trevor Darrell
  6. Frank Hutter
  7. Ming-Hsuan Yang
  8. Yi Yang

In Model Architectures

  1. Esteban Real
  2. Quoc V. Le
  3. Xiaowen Chu
  4. Xin He
  5. Kaiyong Zhao
  6. Alexey Kurakin
  7. Jie Tan
  8. Saurabh Saxena

Citations in the corpus