Self-supervised Label Augmentation via Input Transformations

By Hankook Lee, Sung Ju Hwang, Jinwoo Shin. 2019

223 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Gyuhak Kim
2. Jianping Gou
3. Dacheng Tao
4. Xu-Yao Zhang
5. Jaya Kalla
6. Liyuan Wang
7. Fei Yin
8. Tianlong Chen

Citations in the corpus

• S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning (2023, methodology): “To integrate the stochastic classifiers with self-supervision, for each class, we create four classifier heads corresponding to each of the four rotations as in [22].”
• Knowledge Distillation: A Survey (2020, methodology): “Furthermore, the knowledge transfer from one model to another in knowledge distillation can be extended to other tasks, such as adversarial attacks (Papernot et al. 2016), data augmentation (Lee et al. 2019a; Gordon and Duh 2019), data privacy and security (Wang et al. 2019a).”
• Striking a Balance between Stability and Plasticity for Class-Incremental Learning (2021, methodology): “To address this problem, we propose a SPB-M method by learning multi-perspective knowledge in SPB, which shares the merit of [23] but we do not use self-distillation here.”
• Graph Contrastive Learning Automated (2021, background): “That aligns with the observations in (Lee et al., 2020; Jun et al., 2020), showing that disentan-gling augmented and original feature distributions could have the model benefit more from stronger augmentations.”
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances (2020, methodology): “Note that we do not use the auxiliary classification loss Lcls-SI (4), since the objective already classifies the shifted instances under a self-label augmented [35] space Y × S.”
• CERT: Contrastive Self-supervised Learning for Language Understanding (2020, result): “Some recent works studied self-supervised representation learning based on instance discrimination (Wu et al., 2018) (Chen et al., 2020a) with larger batch sizes and extensive data augmentation (Lee et al., 2019), which achieved results that are comparable with supervised learning.”
• Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation (2021, methodology): “Moreover, there are literatures on a self-knowledge distillation, which trains the student network progressively to distill and to regularize its own knowledge without the pretrained teacher network [43, 35, 18, 32, 39].”
• Prototype Augmentation and Self-Supervision for Incremental Learning (2021, methodology): “Inspired by the natural connection between IL and SSL, we employ the self-supervised method in [32] to investigate SSL in CIL, revealing surprising yet intriguing findings that SSL can boost the performance of CIL significantly.”