How Can Deep Rectifier Networks Achieve Linear Separability and Preserve Distances?

By Senjian An, Farid Boussaïd, Mohammed Bennamoun. 2015

28 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, methodology): “, gradient penalty or norm-preserving activation [1, 2, 12, 28]), and we chose spectral normalization in this work due to its simplicity and its minimal impact on a DNN’s architecture and the optimization dynamics [3, 5, 62].”
Optimization Landscape and Expressivity of Deep CNNs (2017, methodology): “Compared to (An et al., 2015) our Theorem 3.4 is derived for CNNs with a wider range of activation functions.”
Contractive Rectifier Networks for Nonlinear Maximum Margin Classification (2015, background): “The advantages of deep rectifier networks are not only shown in their excellent empirical performance in object recognition [18, 20, 13], face verification [38, 37], speech recognition ([32, 14, 7] and handwritten digit recognition [5], but also justified by a number of theoretical works on their su”
The loss surface and expressivity of deep convolutional neural networks (2017, methodology): “Linear separability in neural networks has been recently studied by An et al. (2015), where the authors show that a two-hidden-layer fully connected network with ReLU activation function can transform any training set to be linearly separable while approximately preserving the distances of the…”
Harnessing noisy Web images for deep representation (2017, background): “A more recent ork ( An et al., 2015 ) proved that a two-layer rectifier network an make any disjoint data linearly separable.”