Multiple-gradient descent algorithm (MGDA) for multiobjective optimization

By Jean-Antoine Desideri. 2012

572 citations recorded by the source corpus.

Subfields: Civil and Structural Engineering, Computational Theory and Mathematics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Civil and Structural Engineering

1. Shan You
2. Fan Lyu
3. Shangchen Du
4. Qing Sun
5. Bo Dai
6. Peter Stone
7. Liang Wan
8. João Monteiro

In Computational Theory and Mathematics

1. Shan You
2. Fan Lyu
3. Shangchen Du
4. Qing Sun
5. Bo Dai
6. Peter Stone
7. Liang Wan
8. João Monteiro

In Management Science and Operations Research

1. Zhe Zhao
2. Ed H. Chi
3. Bo Dai
4. Lichan Hong
5. Qiang Liu
6. Shan You
7. Fan Lyu
8. Shangchen Du

Citations in the corpus

• Exploring Example Influence in Continual Learning (2022, positive): “Rank M with E ( I ∗ ( x )) , x ∈ M ; Inspired by the Multiple-Gradient Descent Algorithm (MGDA) [12], we transform Problem (9) to a min-norm problem.”
• Blind Backdoors in Deep Learning Models (2020, methodology): “Instead, our attack obtains optimal coefficients using Multiple Gradient Descent Algorithm (MGDA) [16].”
• Conflict-Averse Gradient Descent for Multi-task Learning (2021, methodology): “apply the multiple-gradient descent algorithm (MGDA) [6] for MTL, which directly optimizes towards the Pareto set [30].”
• Small Towers Make Big Differences (2020, background): “, 2019) toward bridging this gap is the application of multiple-gradient descent algorithm (Désidéri, 2012) to multi-task learning, which is a gradient-based multi-objective optimization method.”
• Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space (2020, methodology): “Inspired by the success of multi-objective optimization (MOO) problem [5, 25, 4, 20], we formulate the ensemble KD as a MOO problem with each objective corresponding to each teacher, which has been applied to recommender system [18], multi-task learning [26, 16], and multi-agent reinforcement…”
• Multi-Task Learning for Dense Prediction Tasks: A Survey (2020, methodology): “A multiple gradient descent algorithm (MGDA) [71] was proposed in [22] to find a Pareto stationary point.”
• Multi-objective training of Generative Adversarial Networks with multiple discriminators (2019, methodology): “Multiple gradient descent (D´esid´eri, 2012; Sch¨affler et al., 2002; Peitz & Dellnitz, 2018) was proposed for the unconstrained case of multi-objective optimization of F ( x ) assuming a convex, continuously differentiable and smooth f k ( x ) for all k .”
• Efficient Diffusion Training via Min-SNR Weighting Strategy (2023, background): “A more general form of this theorem was first proposed in [11] and we leave a succinct proof in the supplementary material.”
• Multi-Task Learning with Deep Neural Networks: A Survey (2020, methodology): “(Sener and Koltun, 2018) brought gradient-based multi-objective optimization algorithms to the field of deep multi-task learning by extending the well known Multiple Gradient Descent Algorithm (MGDA) (Désidéri, 2012) to a form that scales well to the high dimensionality of deep learning problems.”
• Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout (2020, methodology): “As a subject of study, multitask learning predates deep learning [1, 6], but its power in helping model generalization and transferring information between correlated tasks [30, 48] make it especially relevant in the deep learning era.”