Structured Receptive Fields in CNNs

By Jörn-Henrik Jacobsen, Jan van Gemert, Zhongyu Lou, Arnold M. W. Smeulders. 2016

117 citations recorded by the source corpus.

Learning powerful feature representations with CNNs is hard when training data are limited. Pre-training is one way to overcome this, but it requires large datasets sufficiently similar to the target domain. Another option is to design priors into the model, which can range from tuned hyperparameters to fully engineered representations like Scattering Networks. We combine these ideas into structured receptive field networks, a model which has a fixed filter basis and yet retains the flexibility of CNNs. This flexibility is achieved by expressing receptive fields in CNNs as a weighted sum over a fixed basis which is similar in spirit to Scattering Networks. The key difference is that we learn arbitrary effective filter sets from the basis rather than modeling the filters. This approach explicitly connects classical multiscale image analysis with general CNNs. With structured receptive field networks, we improve considerably over unstructured CNNs for small and medium dataset scenarios as well as over Scattering for large datasets. We validate our findings on ILSVRC2012, Cifar-10, Cifar-100 and MNIST. As a realistic small dataset example, we show state-of-the-art classification results on popular 3D MRI brain-disease datasets where pre-training is difficult due to a lack of large public datasets in a similar domain.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Harmonic Networks: Deep Translation and Rotation Equivariance (2016, methodology): “Usually in our experiments, we use streams of orders 0 and 1, which we found to work well and is justified by the fact that CNN filters tend to contain very little high frequency information [12].”
Dynamic Steerable Blocks in Deep Residual Networks (2017, methodology): “Inspired by the former, Structured Receptive Field Networks are learned from an overcomplete multi-scale frame, effectively improving performance for small datasets due to restricted feature spaces [23].”
Multiscale Hierarchical Convolutional Networks (2017, background): “Other approaches (Jacobsen et al., 2016) represent the filters with few parameters in different bases, instead of imposing tha they have a small spatial support.”
On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location (2020, background): “Other work shows that adding priors increases data efficiency by tying parameters [26], sharing rotation responses [103], and a prior scale-space filter basis [46].”