Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data

By Onur Tasar, Yuliya Tarabalka, Pierre Alliez. 2018

144 citations recorded by the source corpus.

In spite of remarkable success of the convolutional neural networks on semantic segmentation, they suffer from catastrophic forgetting: a significant performance drop for the already learned classes when new classes are added on the data having no annotations for the old classes. We propose an incremental learning methodology, enabling to learn segmenting new classes without hindering dense labeling abilities for the previous classes, although the entire previous data are not accessible. The key points of the proposed approach are adapting the network to learn new as well as old classes on the new training data, and allowing it to remember the previously learned information for the old classes. For adaptation, we keep a frozen copy of the previously trained network, which is used as a memory for the updated network in the absence of annotations for the former classes. The updated network minimizes a loss function, which balances the discrepancy between outputs for the previous classes from the memory and updated networks, and the misclassification rate between outputs for the new classes from the updated network and the new ground-truth. For remembering, we either regularly feed samples from the stored, little fraction of the previous data or use the memory network, depending on whether the new data are collected from completely different geographic areas or from the same city. Our experimental results prove that it is possible to add new classes to the network, while maintaining its performance for the previous classes, despite the whole previous training data are not available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology, Remote Sensing, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Joost van de Weijer
  2. Arthur Douillard
  3. Fabio Cermelli
  4. Matthieu Cord
  5. Xialei Liu
  6. Elisa Ricci
  7. Bartłomiej Twardowski
  8. Andrew D. Bagdanov

In Computer Vision and Pattern Recognition

  1. Joost van de Weijer
  2. Matthieu Cord
  3. Arthur Douillard
  4. Luc Van Gool
  5. Xialei Liu
  6. Elisa Ricci
  7. Fabio Cermelli
  8. Bernt Schiele

In Media Technology

  1. Nassim Ammour
  2. Simone Scardapane
  3. Valerio Marsocci
  4. Joost van de Weijer
  5. Andrew D. Bagdanov
  6. Xialei Liu
  7. Bertrand Le Saux
  8. Elisa Ricci

In Remote Sensing

  1. Pietro Zanuttigh
  2. Bertrand Le Saux
  3. Haifeng Li
  4. Kun Fu
  5. Nassim Ammour
  6. Xian Sun
  7. Umberto Michieli
  8. Chao Tao

In Meta & Continual Learning

  1. Joost van de Weijer
  2. Arthur Douillard
  3. Fabio Cermelli
  4. Xialei Liu
  5. Matthieu Cord
  6. Andrew D. Bagdanov
  7. Bartłomiej Twardowski
  8. Pietro Zanuttigh

Citations in the corpus