Black-box Prompt Tuning with Subspace Learning

• URL: http://arxiv.org/abs/2305.03518v2
• Date: Mon, 17 Jun 2024 03:26:59 GMT
• Title: Black-box Prompt Tuning with Subspace Learning
• Authors: Yuanhang Zheng, Zhixing Tan, Peng Li, Yang Liu,
• Abstract summary: Black-box prompt tuning employs derivative-free optimization algorithms to learn prompts within low-dimensional subspaces. Recent studies reveal that black-box prompt tuning lacks versatility across tasks and Large Language Models (LLMs) We introduce Black-box prompt tuning with Subspace Learning (BSL) to enhance the versatility of black-box prompt tuning.
• Score: 17.310874690694263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Black-box prompt tuning employs derivative-free optimization algorithms to learn prompts within low-dimensional subspaces rather than back-propagating through the network of Large Language Models (LLMs). Recent studies reveal that black-box prompt tuning lacks versatility across tasks and LLMs, which we believe is related to the suboptimal choice of subspaces. In this paper, we introduce Black-box prompt tuning with Subspace Learning (BSL) to enhance the versatility of black-box prompt tuning. Based on the assumption that nearly optimal prompts for similar tasks reside in a common subspace, we propose identifying such subspaces through meta-learning on a collection of similar source tasks. Consequently, for a target task that shares similarities with the source tasks, we expect that optimizing within the identified subspace can yield a prompt that performs well on the target task. Experimental results confirm that our BSL framework consistently achieves competitive performance across various downstream tasks and LLMs.

Related papers

• FedOne: Query-Efficient Federated Learning for Black-box Discrete Prompt Learning [19.60311157467143]
  Black-Box Discrete Prompt Learning is a prompt-tuning method that optimize discrete prompts without accessing model parameters or gradients.<n>We propose the FedOne framework, a federated black-box discrete prompt learning method designed to maximize query efficiency when interacting with cloud-based LLMs.
  arXiv  Detail & Related papers  (2025-06-17T19:21:22Z)
• Diverse Prompts: Illuminating the Prompt Space of Large Language Models with MAP-Elites [2.529560284922988]
  This work introduces an evolutionary approach that combines context-free grammar (CFG) with the MAP-Elites algorithm to explore the prompt space. Our method prioritizes quality and diversity, generating high-performing and structurally varied prompts.
  arXiv  Detail & Related papers  (2025-04-19T17:50:34Z)
• Hyperband-based Bayesian Optimization for Black-box Prompt Selection [15.756224286651237]
  Black-box prompt selection is challenging due to potentially large, search spaces, absence of gradient information, and high evaluation cost of prompts on a validation set.<n>We propose HbBoPs, a novel method that combines a structural-aware deep kernel Gaussian Process with Hyperband as a multi-fidelity scheduler.<n>HbBoPs outperforms state-of-the-art methods in both performance and efficiency.
  arXiv  Detail & Related papers  (2024-12-10T14:42:51Z)
• QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning [58.767866109043055]
  We introduce Query-dependent Prompt Optimization (QPO), which iteratively fine-tune a small pretrained language model to generate optimal prompts tailored to the input queries. We derive insights from offline prompting demonstration data, which already exists in large quantities as a by-product of benchmarking diverse prompts on open-sourced tasks. Experiments on various LLM scales and diverse NLP and math tasks demonstrate the efficacy and cost-efficiency of our method in both zero-shot and few-shot scenarios.
  arXiv  Detail & Related papers  (2024-08-20T03:06:48Z)
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization [73.7779735046424]
  We show that different prompts should be adapted to different Large Language Models (LLM) to enhance their capabilities across various downstream tasks in NLP. We then propose a model-adaptive prompt (MAPO) method that optimize the original prompts for each specific LLM in downstream tasks.
  arXiv  Detail & Related papers  (2024-07-04T18:39:59Z)
• Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement [11.363521189714504]
  We show that large language models (LLMs) are over-sensitive to lexical variations in task instructions. We propose a black-box Combinatorial Optimization framework for Prompt Lexical Enhancement (COPLE)
  arXiv  Detail & Related papers  (2024-05-31T08:53:59Z)
• Prompt Highlighter: Interactive Control for Multi-Modal LLMs [50.830448437285355]
  This study targets a critical aspect of multi-modal LLMs' (LLMs&VLMs) inference: explicit controllable text generation. We introduce a novel inference method, Prompt Highlighter, which enables users to highlight specific prompt spans to interactively control the focus during generation. We find that, during inference, guiding the models with highlighted tokens through the attention weights leads to more desired outputs.
  arXiv  Detail & Related papers  (2023-12-07T13:53:29Z)
• Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning [77.61565726647784]
  We propose a simple black-box search method that first clusters and prunes the search space to focus exclusively on influential prompt tokens. Our findings underscore the critical role of search space design and optimization in enhancing both the usefulness and the efficiency of black-box prompt-based learning.
  arXiv  Detail & Related papers  (2023-10-19T14:25:06Z)
• Distribution-Aware Prompt Tuning for Vision-Language Models [20.02599087680773]
  A key to prompt tuning is the feature space alignment between two modalities via learnable vectors with model parameters fixed. Inspired by this observation, we proposed distribution-aware prompt tuning (DAPT) for vision-language models. Our experiments on 11 benchmark datasets demonstrate that our method significantly improves generalizability.
  arXiv  Detail & Related papers  (2023-09-06T23:49:11Z)
• PromptBoosting: Black-Box Text Classification with Ten Forward Passes [61.38341243907045]
  We describe PromptBoosting, a query-efficient procedure for building a text classifier from a neural language model (LM) without access to the LM's parameters, gradients, or hidden representations. Experiments show that PromptBoosting achieves state-of-the-art performance in multiple black-box few-shot classification tasks, and matches or outperforms full fine-tuning in both few-shot and standard learning paradigms, while training 10x faster than existing black-box methods.
  arXiv  Detail & Related papers  (2022-12-19T06:04:54Z)
• Exploring Low-dimensional Intrinsic Task Subspace via Prompt Tuning [70.76016793057283]
  In this work, we study how pre-trained language models (PLMs) learn universal representations and effectively adapt to broad NLP tasks differing a lot. In experiments, we study diverse few-shot NLP tasks and surprisingly find that in a 5-dimensional subspace found with 100 random tasks, by only tuning 5 free parameters, we can recover 87% and 65% of the full prompt tuning performance.
  arXiv  Detail & Related papers  (2021-10-15T05:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.