Training Small Networks for Scene Classification of Remote Sensing Images via Knowledge Distillation

By Guanzhou Chen, Xiaodong Zhang, Xiaoliang Tan, Yufeng Cheng, Fan Dai, Kun Zhu, Yuanfu Gong, Qing Wang. 2018

76 citations recorded by the source corpus.

Scene classification, aiming to identify the land-cover categories of remotely sensed image patches, is now a fundamental task in the remote sensing image analysis field. Deep-learning-model-based algorithms are widely applied in scene classification and achieve remarkable performance, but these high-level methods are computationally expensive and time-consuming. Consequently in this paper, we introduce a knowledge distillation framework, currently a mainstream model compression method, into remote sensing scene classification to improve the performance of smaller and shallower network models. Our knowledge distillation training method makes the high-temperature softmax output of a small and shallow student model match the large and deep teacher model. In our experiments, we evaluate knowledge distillation training method for remote sensing scene classification on four public datasets: AID dataset, UCMerced dataset, NWPU-RESISC dataset, and EuroSAT dataset. Results show that our proposed training method was effective and increased overall accuracy (3% in AID experiments, 5% in UCMerced experiments, 1% in NWPU-RESISC and EuroSAT experiments) for small and shallow models. We further explored the performance of the student model on small and unbalanced datasets. Our findings indicate that knowledge distillation can improve the performance of small network models on datasets with lower spatial resolution images, numerous categories, as well as fewer training samples.

Subfields: Computer Vision and Pattern Recognition, Ecology, Media Technology, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Junwei Han
2. Gong Cheng
3. Guisong Xia
4. Lei Guo
5. Xingxing Xie
6. Peicheng Zhou
7. Andreas R. Dengel
8. Benjamin Bischke

In Ecology

1. Lei Guo
2. Andreas R. Dengel
3. Damian Borth
4. Gong Cheng
5. Patrick Helber
6. Benjamin Bischke
7. Guisong Xia
8. Junwei Han

In Media Technology

1. Gong Cheng
2. Junwei Han
3. Guisong Xia
4. Lei Guo
5. Xingxing Xie
6. Peicheng Zhou
7. Andreas R. Dengel
8. Damian Borth

In Efficiency & Compression

1. Lei Guo
2. Gong Cheng
3. Junwei Han

Citations in the corpus

• EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification (2017, background): “The effectiveness of these approaches was successfully demonstrated in [32]–[34].”
• Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities (2020, background): “sign compact and lightweight scene classification models [120], [121].”