A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications

By Ankur Sinha, Pekka Malo, Kalyanmoy Deb. 2017

909 citations recorded by the source corpus.

Bilevel optimization is defined as a mathematical program, where an optimization problem contains another optimization problem as a constraint. These problems have received significant attention from the mathematical programming community. Only limited work exists on bilevel problems using evolutionary computation techniques; however, recently there has been an increasing interest due to the proliferation of practical applications and the potential of evolutionary algorithms in tackling these problems. This paper provides a comprehensive review on bilevel optimization from the basic principles to solution strategies; both classical and evolutionary. A number of potential application problems are also discussed. To offer the readers insights on the prominent developments in the field of bilevel optimization, we have performed an automated text-analysis of an extended list of papers published on bilevel optimization to date. This paper should motivate evolutionary computation researchers to pay more attention to this practical yet challenging area.

Subfields: Computational Theory and Mathematics, Control and Systems Engineering, Ocean Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Control and Systems Engineering

In Ocean Engineering

Citations in the corpus

Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems (2020, methodology): “2 and (3)] is derived using a dedicated bilevel optimization algorithm [31], where the upper level searches the template of the nonlinear rule and the corresponding lower level focuses at estimating optimal values of weights/coefﬁcients for optimal split of data present in the conditional node.”
Adaptive Aggregation Networks for Class-Incremental Learning (2020, methodology): “Ideally, in the i-th phase, the CIL system aims to learn the optimal αi and [φi, ηi] that minimize the classification loss on all training samples seen so far, i.e., Di ∪ D0:i−1, so the ideal BOP can be formulated as, min αi L(αi, φ∗i , η∗i ;D0:i−1 ∪ Di) (4a) s.t. [φ∗i , η ∗ i ] = argmin [φi,ηi] L(α”
Mnemonics Training: Multi-Class Incremental Learning Without Forgetting (2020, methodology): “In particular, for each single phase, we perform a local BOP that aims to distill the knowledge of new class data into the exemplars.”
Meta-Learning in Neural Networks: A Survey (2020, background): “Bilevel optimization [42] refers to a hierarchical optimization problem, where one optimization contains another optimization as a constraint [25], [43].”
Approximation Methods for Bilevel Programming (2018, background): “Two surveys of existing works can be found in [6, 18].”
λOpt: Learn to Regularize Recommender Models in Finer Levels (2019, background): “Essentially, this can be formulated as a nested optimization problem [27] or bi-level optimization problem [29]:”
Interpretable Rule Discovery Through Bilevel Optimization of Split-Rules of Nonlinear Decision Trees for Classification Problems (2020, methodology): “This hierarchical structure of the variables motivates us to employ a bilevel optimization approach [37] to handle the two types of variables.”
Fast and Practical Neural Architecture Search (2019, methodology): “We apply the bi-level optimization [28], which has two levels of optimization formally written as This function can be efﬁciently optimized as discussed in [28].”
Meta-Aggregating Networks for Class-Incremental Learning (2020, methodology): “We formulate these two steps in a bilevel optimization program (BOP) and conduct the optimizations alternatively, i.e., update network parameters with aggregating weights ﬁxed, and then switch (Sinha et al., 2018; MacKay et al., 2019; Liu et al., 2020).”