The unreasonable effectiveness of Batch-Norm statistics in addressing catastrophic forgetting across medical institutions

By Sharut Gupta, Praveer Singh, Ken Chang, M. Aggarwal, N. Arun, Liangqiong Qu, K. Hoebel, J. Patel, M. Gidwani, Ashwin Vaswani and 2 more. 2020

3 citations recorded by the source corpus.

Model brittleness is a primary concern when deploying deep learning models in medical settings owing to inter-institution variations, like patient demographics and intra-institution variation, such as multiple scanner types. While simply training on the combined datasets is fraught with data privacy limitations, fine-tuning the model on subsequent institutions after training it on the original institution results in a decrease in performance on the original dataset, a phenomenon called catastrophic forgetting. In this paper, we investigate trade-off between model refinement and retention of previously learned knowledge and subsequently address catastrophic forgetting for the assessment of mammographic breast density. More specifically, we propose a simple yet effective approach, adapting Elastic weight consolidation (EWC) using the global batch normalization (BN) statistics of the original dataset. The results of this study provide guidance for the deployment of clinical deep learning models where continuous learning is needed for domain expansion.

Subfields: Artificial Intelligence, Family Practice

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Continual Learning via Local Module Composition (2021, background): “Specifically, in continual learning when testing on the previously seen tasks after new tasks have been learned, the batch norm will change its statistics resulting in forgetting even if the parameters of the network have not been changed [31].”