Bilevel Optimization with Nonsmooth Lower Level Problems

By Peter Ochs, René Ranftl, Thomas Brox, Thomas Pock. 2015

87 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization (2016, methodology): “In this technical report we focus on ﬁrst-order gradient based techniques, which have become very important in machine learning and computer vision with the wide spread adoption of deep neural network models [LeCun et al., 2015, Schmidhuber, 2015, Krizhevsky et al., 2012].”
Optimizing Millions of Hyperparameters by Implicit Differentiation (2019, methodology): “The IFT has been used for optimization in nested optimization problems [27, 28], backpropagating through arbitrarily long RNNs [22], or even efficient k-fold cross-validation [29].”
Modern regularization methods for inverse problems (2018, methodology): “These cover the finitedimensional Markov random field models proposed in [325, 346, 143, 124, 334], the optimal model design approaches in [199, 198, 65, 40], the optimal regularization parameter estimation in variational regularization [89, 128, 137, 138, 90, 127], to training optimal operators in ”
Techniques for Gradient-Based Bilevel Optimization with Non-smooth Lower Level Problems (2016, methodology): “Instead of repeating the experiment, we refer to [29] for the experiment.”
Deep Declarative Networks (2019, methodology): “[42] extend bi-level optimization to handle non-smooth lower-level problems and apply their method to image segmentation tasks.”
Deep Network Flow for Multi-object Tracking (2017, background): “…can be rearranged to The ﬁnal derivative can then be written as To fully deﬁne (10), we provide the second derivative of P w.r.t. x , which is given as In the supplemental material we show that (10) is equivalent to a generic solution provided in [36] and that B ∂ 2 P ∂ x 2 B is always invertible.”
Learning End-to-end Video Classification with Rank-Pooling (2016, background): “Fortunately, when the lower level objective is twice differentiable we can compute the gradient of the argmin function as other authors have also observed (Ochs et al., 2015; Domke, 2012; Do et al., 2007) and the following lemmas show.”
Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection (2022, methodology): “Following the truism Stackelberg’s theory [14, 18, 27], we formulate the detection-oriented fusion as a bilevel optimization model: where L d is the detection-speciﬁc training loss and Ψ denotes a detection network with learnable parameters ω d . f ( · ) is an energy-based ﬁdelity term containing…”
Bilevel approaches for learning of variational imaging models (2015, methodology): “Two main directions may be considered in this context: Solving the original problem via optimisation methods [18, 32, 63], and solving the full optimality system of equations [54, 25].”