Lipschitz Properties for Deep Convolutional Networks

By Radu V. Balan, Maneesh Kumar Singh, Dongmian Zou. 2017

52 citations recorded by the source corpus.

In this paper we discuss the stability properties of convolutional neural networks. Convolutional neural networks are widely used in machine learning. In classification they are mainly used as feature extractors. Ideally, we expect similar features when the inputs are from the same class. That is, we hope to see a small change in the feature vector with respect to a deformation on the input signal. This can be established mathematically, and the key step is to derive the Lipschitz properties. Further, we establish that the stability results can be extended for more general networks. We give a formula for computing the Lipschitz bound, and compare it with other methods to show it is closer to the optimal value.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Regularisation of neural networks by enforcing Lipschitz continuity (2018, methodology): “Balan et al. (2017) provide a means for computing an upper bound to the Lipschitz constant of a restricted class of neural networks known as scattering networks.”
Concolic Testing for Deep Neural Networks (2018, methodology): “In this section, we use DR+ formulas to express several important requirements for DNNs, including Lipschitz continuity [1, 3, 20, 29, 30] and test criteria [15, 18, 23].”
Lipschitz Continuity in Model-based Reinforcement Learning (2018, background): “Note that the question of bounding the Lipschitz constant of a neural network has been studied previously for convolutional networks (Arjovsky et al., 2017; Balan et al., 2017), but, to the best of our knowledge, we are the first to quantify the bounds presented in Table 2.”
Property-driven Training: All You (N)Ever Wanted to Know About (2021, background): “More recently, a third competing deﬁnition of robustness has been proposed: Lipschitz robustness (Balan, Singh, and Zou 2018).”
Lipschitz regularity of deep neural networks: analysis and efficient estimation (2018, methodology): “ion 4.3], as the product of the spectral norms of linear layers (a special case of our generic algorithm, see Proposition 1). More recently, the Lipschitz constant of scatter networks was analyzed in [19]. Unfortunately, this analysis does not extend to more general architectures. Our aim in this pa”
Avoiding Resentment Via Monotonic Fairness (2019, background): “continuous functions, although the Lipschitz constant may be large [38, 20, 3].”
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks (2019, background): “Bounding Lipschitz constants for the specific case of convolutional neural networks (CNNs) has also been addressed in [5, 44, 6].”
On Lipschitz Bounds of General Convolutional Neural Networks (2018, background): “8: The toy example that also appears in [21].”