Capturing the learning curves of generic features maps for realistic data sets with a teacher-student model

By Bruno Loureiro, Cédric Gerbelot, Hugo Cui, Sebastian Goldt, Florent Krzakala, M. M'ezard, Lenka Zdeborov'a. 2021

35 citations recorded by the source corpus.

Subfields: Optimization & Training, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

1. Ankur Bapna
2. Xavier García
3. Colin Cherry
4. Ciprian Chelba
5. Maxim Krikun
6. Alessandro Favero
7. Francesco Cagnetta
8. Orhan Firat

In Scaling & Learning Theory

1. Alessandro Favero
2. Francesco Cagnetta
3. Matthieu Wyart
4. Behrooz Ghorbani

Citations in the corpus

• Scaling Laws for Neural Machine Translation (2021, background): “…theoretical understanding of the factors governing the quality and training dynamics of large neural networks has also being developing (Advani et al., 2020; Rosenfeld et al., 2019; Geiger et al., 2020; Ghorbani et al., 2021; Cho, 2020; Hutter, 2021; Bahri et al., 2021; Loureiro et al., 2021).”
• Locality defeats the curse of dimensionality in convolutional teacher–student scenarios (2021, background): “Although the general modal decomposition cannot be proven rigorously in the ridgeless limit [19, 21], additional results are available when the target functions are Gaussian random fields with covariance specified by a teacher kernel: • Equation (8) can be proven rigorously [4] if teacher and studen”