Visual Prompting: Modifying Pixel Space to Adapt Pre-trained Models

By Hyojin Bahng, Ali Jahanian, S. Sankaranarayanan, Phillip Isola. 2022

121 citations recorded by the source corpus.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

Learning to Prompt for Vision-Language Models (2021, background): “It would also be interesting to investigate more generic adaptation methods for mega-size vision models (Jia et al., 2022; Bahng et al., 2022; Gao et al., 2021).”
Visual Prompt Tuning (2022, methodology): “Due to the disparity between vision and language modalities, in this paper we ask: can the same method can be applied successfully to image encoders? We are the first work (see related concurrent works [52,62,11,2]) to tackle this question and investigate the generality and feasibility of visual pro”
S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, methodology): “There have been emerging a bunch of prompting methods, such as [35, 38, 26, 76, 77, 3, 24, 66, 14], to name a few 6 .”
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, unknown): “Inspired by the success of prompt learning in computer vision [314], ProSFDA [141] adds a learnable image perturbation to all target data so that the BN statistics can be aligned with those stored in the source model.”
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition (2022, methodology): “, [4] aimed to adapt pre-trained models by modifying raw input pixel space.”
Improving Visual Prompt Tuning for Self-supervised Vision Transformers (2023, background): “Recently, considerable research has been conducted on tuning large pretrained models in a parameter-efficient manner (Jia et al., 2022; Houlsby et al., 2019; Cai et al., 2020; Chen et al., 2022; Bahng et al., 2022; Li & Liang, 2021; Lester et al., 2021; Huang et al., 2023).”
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning for Action Recognition (2022, background): “More recently, [29, 4, 84] introduce the idea of prompt learning to visual backbones.”
Self-regulating Prompts: Foundational Model Adaptation without Forgetting (2023, background): “Prompt Learning for CLIP: Prompt learning approaches append learnable prompt tokens at either the text [59, 58] encoder or image [1] encoder.”