Task-Oriented Feature Distillation

73 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector (2021, neutral): “Instead, we use the feature distillation loss [45], which is formulated as where F new / F old are the feature extractors of −→ W t / W 1”
Knowledge Distillation: A Survey (2020, methodology): “…mutual teacher–student networks (Thoker and Gall 2019), multiple stream networks (Garcia et al. 2018), spatiotemporal distilled dense-connectivity network (Hao and Zhang 2019), graph distillation (Luo et al. 2018) and multi-teacher to multi-student networks (Wu et al. 2019b; Zhang et al. 2020c).”