Black-box Prompt Learning for Pre-trained Language Models

By Shizhe Diao, Xuechun Li, Yong Lin, Zhichao Huang, Xiao Zhou, Tong Zhang. 2022

90 citations recorded by the source corpus.

The increasing scale of general-purpose Pre-trained Language Models (PLMs) necessitates the study of more efficient adaptation across different downstream tasks. In this paper, we establish a Black-box Discrete Prompt Learning (BDPL) to resonate with pragmatic interactions between the cloud infrastructure and edge devices. Particularly, instead of fine-tuning the model in the cloud, we adapt PLMs by prompt learning, which efficiently optimizes only a few parameters of the discrete prompts. Moreover, we consider the scenario that we do not have access to the parameters and gradients of the pre-trained models, except for its outputs given inputs. This black-box setting secures the cloud infrastructure from potential attack and misuse to cause a single-point failure, which is preferable to the white-box counterpart by current infrastructures. Under this black-box constraint, we apply a variance-reduced policy gradient algorithm to estimate the gradients of parameters in the categorical distribution of each discrete prompt. In light of our method, the user devices can efficiently tune their tasks by querying the PLMs bounded by a range of API calls. Our experiments on RoBERTa and GPT-3 demonstrate that the proposed algorithm achieves significant improvement on eight benchmarks in a cloud-device collaboration manner. Finally, we conduct in-depth case studies to comprehensively analyze our method in terms of various data sizes, prompt lengths, training budgets, optimization objectives, prompt transferability, and explanations of the learned prompts. Our code will be available at https://github.com/shizhediao/Black-Box-Prompt-Learning.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, methodology): “[57] recently used black-box optimization to train prompt embeddings without requiring gradients.”
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models (2022, background): “Recently, Sun et al. (2022) and Diao et al. (2022) explored blackbox tuning, which trains the prompt without gradient back-propagation.”
Fine-Tuning Language Models with Just Forward Passes (2023, background): “Other works in “black-box tuning” of LMs focus on optimizing discrete prompts without updating the model, either via reinforcement learning [16, 25, 29], ensemble [45], or iterative search [78].”
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, background): “, 2021), require expensive gradient information (Sun et al., 2022; Diao et al., 2022) and are incompatible for reuse across models due to mismatched latent spaces (Su et al.”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “Therefore, how to enable effective knowledge injection for black-box LLMs is still an open question for us to explore [255], [256].”