Learning to Teach with Dynamic Loss Functions

By Lijun Wu, Fei Tian, Yingce Xia, Yang Fan, Tao Qin, J. Lai, Tie-Yan Liu. 2018

126 citations recorded by the source corpus.

Teaching is critical to human society: it is with teaching that prospective students are educated and human civilization can be inherited and advanced. A good teacher not only provides his/her students with qualified teaching materials (e.g., textbooks), but also sets up appropriate learning objectives (e.g., course projects and exams) considering different situations of a student. When it comes to artificial intelligence, treating machine learning models as students, the loss functions that are optimized act as perfect counterparts of the learning objective set by the teacher. In this work, we explore the possibility of imitating human teaching behaviors by dynamically and automatically outputting appropriate loss functions to train machine learning models. Different from typical learning settings in which the loss function of a machine learning model is predefined and fixed, in our framework, the loss function of a machine learning model (we call it student) is defined by another machine learning model (we call it teacher). The ultimate goal of teacher model is cultivating the student to have better performance measured on development dataset. Towards that end, similar to human teaching, the teacher, a parametric model, dynamically outputs different loss functions that will be used and optimized by its student model at different training stages. We develop an efficient learning method for the teacher model that makes gradient based optimization possible, exempt of the ineffective solutions such as policy optimization. We name our method as "learning to teach with dynamic loss functions" (L2T-DLF for short). Extensive experiments on real world tasks including image classification and neural machine translation demonstrate that our method significantly improves the quality of various student models.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Tie-Yan Liu
3. Sergey Levine
4. Yingce Xia
5. Xu Tan
6. Timothy M. Hospedales
7. Chelsea Finn
8. Fei Tian

Citations in the corpus

• Neural Architecture Optimization (2018, methodology): “Third, we plan to design better neural models from the view of teaching and learning to teach [13, 44].”
• A Survey on Curriculum Learning (2021, methodology): “Beyond loss weights, some other works [97], [124] also focus on learning dynamic loss function as a whole, which complies with the most general definition of CL in Sec.”
• Human Pose Regression with Residual Log-likelihood Estimation (2021, methodology): “[63] using a teacher model to dynamically change the loss function of the student model.”
• Curriculum Temperature for Knowledge Distillation (2022, methodology): “This training strategy has been widely applied in various domains, such as computer vision (Wu et al. 2018; Sinha, Garg, and Larochelle 2020) and natural language processing (Platanios et al.”
• Online Meta-Critic Learning for Off-Policy Actor-Critic Methods (2020, background): “Loss learning has been exploited in ‘learning to teach’ (Wu et al., 2018) and surrogate loss learning (Huang et al.”
• Curriculum Learning: A Survey (2021, methodology): “NIPS 2018 [58] CV, NLP classification, machine translation teacherstudent loss iterative model”
• A Competence-aware Curriculum for Visual Concepts Learning via Question Answering (2020, background): “the diculty of each question. 2.2 Curriculum Learning and Machine Teaching The competence-aware curriculum in our work is related to curriculum learning [7,54,56,20,51,45,21,48] and machine teaching [61,62,39,11,41,15,58]. Curriculum learning is rstly proposed by Bengio et al. [7] and demonstrates t”
• Meta Learning via Learned Loss (2019, background): “1For simple gradient descent: h(θ,∇θL(y, fθ(x)) = θ − ψ∇θL(y, fθ(x)) Closest to our method are the works on evolved policy gradients [11], teacher networks [32], meta-critics [27] and meta-gradient RL [33].”
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting (2019, methodology): “The parameters contained in MW-Net can be optimized by using the meta learning idea [32, 33, 34, 35].”
• Learning to Generalize from Sparse and Underspecified Rewards (2019, methodology): “Recent work has also applied meta-learning to reweight learning examples (Ren et al., 2018) to enable robust supervised learning with noisy labels and learning dynamic loss functions (Wu et al., 2018) for improving generalization performance in supervised learning.”
• Learning to Reweight with Deep Interactions (2021, methodology): “…from the student model and the validation set as inputs and outputs some signals to guide the training of the student model, e.g., adjusting the weights of training data (Fan et al. 2018; Shu et al. 2019; Jiang et al. 2018; Ren et al. 2018), generating better loss functions (Wu et al. 2018), etc.”