Transfusion: Understanding Transfer Learning with Applications to Medical Imaging

By Maithra Raghu, Chiyuan Zhang, Jon M. Kleinberg, Samy Bengio. 2019

95 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning, Medical Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Big Transfer (BiT): General Visual Representation Learning (2019, methodology): “A network is trained once on a large, generic dataset, and its weights are then used to initialize subsequent tasks which can be solved with fewer data points, and less compute [40,44,14].”
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, methodology): “For example by learning: architectures Zoph & Le (2017), optimization algorithms Bello et al. (2017), initialization distributions Raghu et al. (2019), or pre-trained data representations.”
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML (2019, methodology): “[21] Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio.”
Investigating Transferability in Pretrained Language Models (2020, methodology): “In their reinitialization experiments on a convolutional neural network for medical images, Raghu et al. (2019) found that a 5x larger rate on the reinitialized layers enabled their model to achieve higher ﬁnetuning accuracy.”
Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks (2019, background): “Data representation is important for robust model learning (Bengio et al., 2013); however, many of the structures that make learning effective in popular ML benchmarks (e.g., Gabor filters for computer vision given natural images) do not map over to clinical equivalents (Raghu et al., 2019).”
Disentangling trainability and generalization in deep learning (2019, background): “…a dizzying array of tasks including vision (Cubuk et al., 2019), speech recognition (Park et al., 2019), machine translation (Bahdanau et al., 2014), chemical property prediction (Gilmer et al., 2017), diagnosing medical conditions (Raghu et al., 2019), and playing games (Silver et al., 2018).”
Large Scale Learning of General Visual Representations for Transfer (2019, background): “Although the pre-training phase is expensive, downstream tasks often require fewer training iterations [37,12].”
Deep semantic segmentation of natural and medical images: a review (2019, background): “Therefore, a proper research direction would be along Raghu et al.’s work [116] on image classiﬁcation models, studying the risks of us-ing non-medical pre-trained models for medical image segmentation.”
In-domain representation learning for remote sensing (2019, background): “These approaches extend beyond natural images and have been used the in medical domain, leading Raghu et al. (2019) to question on how they work.”
Rethinking the Hyperparameters for Fine-tuning (2020, unknown): “Raghu et al. (2019) ﬁnd that transfer learning has negligible performance boost on medical imaging applications, but speed up the convergence signiﬁcantly.”