Test-Time Training with Self-Supervision for Generalization under Distribution Shifts

By Yu Sun, X. Wang, Zhuang Liu, John Miller, Alexei A. Efros, Moritz Hardt. 2019

1,341 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…trained models can suffer at segmenting nighttime driving scenes (Dai and Van Gool, 2018), detecting relevant objects in new or challenging locations and settings (Yu et al., 2020; Sun et al., 2020), and, as discussed earlier, detecting pedestrians with darker skin tones (Wilson et al., 2019).”
Transformer Memory as a Differentiable Search Index (2022, background): “As one example, since indexing is now a special case of model training, incrementally updating an index becomes a special case of model updating (Sun et al., 2020).”
Learning Improvement Heuristics for Solving Routing Problems (2019, background): “This is reasonable because, in general, achieving good out-ofdistribution generalization is very hard for ML models [27], [53], [54].”
Continual Test-Time Domain Adaptation (2022, methodology): “standard domain adaptation Yes stationary Yes No standard test-time training [54] Yes stationary Yes (aux task) Yes fully test-time adaptation [61] No stationary No (pre-trained) Yes”
NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation (2022, methodology): “Second, while some recent TTA methods leverage augmentations to improve performance at the cost of additional forwarding passes [35, 44], NOTE requires only a single forwarding pass.”
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models (2022, background): “Another choice is to update the feature extractor while freezing the prediction module [44, 53].”
Towards Stable Test-Time Adaptation in Dynamic Wild World (2023, methodology): “Recently, numerous test-time adaptation (TTA) methods (Sun et al., 2020; Wang et al., 2021; Iwa-sawa & Matsuo, 2021; Bartler et al., 2022) have been proposed to conquer the above domain shifts by online updating a model on the test data, which include two main categories, i.e. , Test-Time Training…”
Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift (2020, methodology): “TTT offers additional support for this hypothesis: this method also works well across most metrics, in line with previous results on the original versions of the corrupted image benchmarks [63], but works notably well for rotated MNIST.”
Learning to (Learn at Test Time): RNNs with Expressive Hidden States (2024, positive): “The core idea of Test-Time Training (TTT) is that each test instance defines its own learning problem, where this test instance alone is the target of generalization [72].”
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey (2022, background): “Some recent works apply self-supervised techniques on the unlabeled test-data [293], [294], [295], which is useful to overcome possible shifts in the data distribution [296].”
TTT++: When Does Self-Supervised Test-Time Training Fail or Thrive? (2021, methodology): “More closely related to ours, [6] proposed test-time training through self-supervised learning, e.g. , predicting the type of image rotations.”
Prompt Consistency for Zero-Shot Task Generalization (2022, methodology): “This is similar to traditional settings where training and test data are different; and (2) test-time tuning (Sun et al., 2020; Wang et al., 2021): we directly adapt the PLM on the test data.”
Sketch3T: Test-Time Training for Zero-Shot SBIR (2022, methodology): “Following [14] we keep 30 random classes for testing, while 220 training classes are split randomly as 150 for meta-train and 70 for meta-test.”
MEMO: Test Time Robustness via Adaptation and Augmentation (2021, methodology): “We do not compare to TTT for the ImageNet experiments due to the computational demands of training state-of-the-art models and because Sun et al. [44] do not report competitive ImageNet results.”
Hand-Object Contact Consistency Reasoning for Human Grasps Generation (2021, methodology): “Sun et al. [44] extended the test-time adaptation idea to more general applications with a joint training framework of an image recognition task and a self-supervised task.”
Space-Time Correspondence as a Contrastive Random Walk (2020, background): “A key beneﬁt of self-supervised learning is that, because there is no reliance on labeled data, training need not be limited to the training phase, but can continue during deployment [3, 56, 75].”
Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift (2020, background): “A common theme in these approaches is that they assume access to a set of unlabeled samples from the test set at training time [Sun et al., 2019], whereas our focus is to apply an intervention at test time, after a model has been trained.”