Efficient Perturbation Inference and Expandable Network for continual learning

By Fei Du, Yun Yang, Ziyuan Zhao, Zeng Zeng. 2022

24 citations recorded by the source corpus.

Although humans are capable of learning new tasks without forgetting previous ones, most neural networks fail to do so because learning new tasks could override the knowledge acquired from previous data. In this work, we alleviate this issue by proposing a novel Efficient Perturbation Inference and Expandable Network (EPIE-Net), which dynamically expands lightweight task-specific decoders for new classes and utilizes a mixed-label uncertainty strategy to improve the robustness. Moreover, we calculate the average probability of perturbed samples at inference, which can generally improve the performance of the model. Experimental results show that our method consistently outperforms other methods with fewer parameters in class incremental learning benchmarks. For example, on the CIFAR-100 10 steps setup, our method achieves an average accuracy of 76.33% and the last accuracy of 65.93% within only 3.46M average parameters.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Hang Su
  2. Jun Zhu
  3. Liyuan Wang
  4. Xingxing Zhang
  5. Tianyu Pang
  6. Yinpeng Dong
  7. Vamsi Krishna Ithapu
  8. Mirella Lapata

In Computer Vision and Pattern Recognition

  1. Hang Su
  2. Jun Zhu
  3. Liyuan Wang
  4. Xingxing Zhang
  5. Yinpeng Dong
  6. Vamsi Krishna Ithapu
  7. Alan L. Yuille
  8. Tianyu Pang

In Meta & Continual Learning

  1. Xingxing Zhang
  2. Liyuan Wang
  3. Hang Su
  4. Jun Zhu
  5. Simone Scardapane
  6. Valerio Marsocci
  7. Liyuan Wang
  8. Hongwei Yan

Citations in the corpus