Black-Box Tuning for Language-Model-as-a-Service

By Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, Xipeng Qiu. 2022

354 citations recorded by the source corpus.

Extremely large pre-trained language models (PTMs) such as GPT-3 are usually released as a service. It allows users to design task-specific prompts to query the PTMs through some black-box APIs. In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable. Can we optimize the task prompts by only accessing the model inference APIs? This paper proposes the black-box tuning framework to optimize the continuous prompt prepended to the input text via derivative-free optimization. Instead of optimizing in the original high-dimensional prompt space, which is intractable for traditional derivative-free optimization, we perform optimization in a randomly generated subspace due to the low intrinsic dimensionality of large PTMs. The experimental results show that the black-box tuning with RoBERTa on a few labeled samples not only significantly outperforms manual prompt and GPT-3's in-context learning, but also surpasses the gradient-based counterparts, i.e., prompt tuning and full model tuning.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

CLIP-Adapter: Better Vision-Language Models with Feature Adapters (2021, neutral): “Some other methods, e.g., black-box tuning [56], ladder side-tuning [57], sparse structure search[26], and Scaling&Shifting [38] Prompt Design.”
Can We Edit Factual Knowledge by In-Context Learning? (2023, background): “Previous studies have shown that when a large-scale LM (LLM) is deployed as a black-box service (Sun et al., 2022), a minor modification to its parameters could dramatically influence its behavior for end users.”
DyLoRA: Parameter-Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation (2022, methodology): “PETs can be alternatives to the compres-a sion methods, especially when we would like to use the full capacity of the large pretrained models with light training efforts (such as the language-model-as-a-service scenario (Sun et al., 2022)).”
Black-box Prompt Learning for Pre-trained Language Models (2022, background): “• BBT (Sun et al., 2022): a frozen ROBERTA- large model with continuous prompts that are optimized by covariance matrix adaptation evolution strategy (black-box).”
Fine-Tuning Language Models with Just Forward Passes (2023, methodology): “ZO significantly outperform BBTv2.”
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition (2023, methodology): “Inspired by previous work (Sun et al., 2022), we utilize a black-box optimization technique to find the optimal w .”
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers (2024, neutral): “To adapt to downstream tasks, simply adding an instruction to the input text, also called discrete prompt, steers LLMs to carry out the desired task with negligible impact on computational cost (Liu et al., 2023).”
The rise and potential of large language model based agents: a survey (2025, neutral): “Therefore, users construct prompts to query models through APIs, a method referred to as language model as a service (LMaaS) [574].”
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023, background): “The impressive performance of GPT-3 and other LLMs (Scao et al., 2022; Tay et al., 2022; Thoppilan et al., 2022; Fedus et al., 2021; Hoffmann et al., 2022; Zeng et al., 2022) in few-shot learning scenarios is a major finding as this helps LLMs to be more efficient, making it possible to use LM-as-a-”
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, methodology): “For Prompt Tuning (Lester et al., 2021), we replace the Manual Prompt tokens with five soft to-kens in the same positions for fair comparison, and optimize them using Adam optimizer with learning rate 1 × 10 − 2 and batch size 16 for 400 epochs.”
Unifying Large Language Models and Knowledge Graphs: A Roadmap (2023, background): “Therefore, how to enable effective knowledge injection for black-box LLMs is still an open question for us to explore [255], [256].”
Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs (2023, methodology): “For instance, ChatGPT [45], along with its API, solely provides a text-based interface.”