Tent: Fully Test-Time Adaptation by Entropy Minimization

By Dequan Wang, Evan Shelhamer, Shaoteng Liu, Barbara Olshausen, Trevor Darrell. 2021

1,933 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

Learning to Prompt for Vision-Language Models (2021, background): “Nevertheless, the simplicity of CoOp allows easy extension for future work and there remain many interesting questions to explore, such as cross-dataset transfer (Zhou et al., 2022a) and test-time adaptation (Wang et al., 2020).”
Unsolved Problems in ML Safety (2021, background): “[195] Dequan Wang, Evan Shelhamer, Shaoteng Liu, B.”
Git Re-Basin: Merging Models modulo Permutation Symmetries (2022, methodology): “Therefore, we recommend the recalculation of batch statistics after merging models (Izmailov et al., 2018; Wortsman et al., 2021; Maddox et al., 2019; Wang et al., 2021).”
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models (2022, methodology): “TENT [43] proposes a test-time objective by minimizing the entropy of the batch-wise prediction probability distributions.”
Domain Generalization: A Survey (2021, neutral): “tion [159], blurs the boundary betweenDAandDG.”
Efficient Test-Time Model Adaptation without Forgetting (2022, methodology): “Here, the entropy loss E ( · ) is computed over a batch of samples each time (similar to Tent (Wang et al., 2021)) to avoid a trivial solution, i.e. , assigning all probability to the most probable class.”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “Tent [9] and RNCR [491] propose replacing the fixed BN statistics (i.”
Sketch3T: Test-Time Training for Zero-Shot SBIR (2022, methodology): “As an auxiliary task during training, it should improve robustness of the primary task [20], like rotation prediction [58] or via entropy minimisation [62].”
Prompt Consistency for Zero-Shot Task Generalization (2022, methodology): “This is similar to traditional settings where training and test data are different; and (2) test-time tuning (Sun et al., 2020; Wang et al., 2021): we directly adapt the PLM on the test data.”
MEMO: Test Time Robustness via Adaptation and Augmentation (2021, methodology): “We also compare to Tent [36], which adapts by minimizing conditional entropy, and BN adaptation – both of these methods can be used with pretrained models but require multiple test inputs (or even the entire test set) for adaptation.”
Normalization Techniques in Training DNNs: Methodology, Analysis and Application (2020, background): “[244] also propose fully test-time adaptation (Tent) for domain adaption or image corruption.”
NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation (2022, methodology): “Temporally correlated distributions lead to an undesirable bias in p ( y ) , and thus adaptation with a batch of consecutive test samples negatively impacts the adaptation objective, such as entropy minimization [41].”
TTT++: When Does Self-Supervised Test-Time Training Fail or Thrive? (2021, methodology): “However, it was shown inferior to other families of test-time adaptation methods [7, 8].”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Some recent works apply self-supervised techniques on the unlabeled test-data [293], [294], [295], which is useful to overcome possible shifts in the data distribution [296].”
Contrastive Test-Time Adaptation (2022, methodology): “To address these difficulties, existing works have leveraged image/feature generation [23, 27], class prototypes [28, 57], entropy minimization [28, 56], self-training or pseudo labeling [28], and self-supervised auxiliary task training [50].”