Reducing Retraining by Recycling Parameter-Efficient Prompts

• URL: http://arxiv.org/abs/2208.05577v1
• Date: Wed, 10 Aug 2022 22:10:53 GMT
• Title: Reducing Retraining by Recycling Parameter-Efficient Prompts
• Abstract summary: We show that recycling between models is possible (our best settings are able to successfully recycle $88.9%$ of prompts, producing a prompt that out-performs baselines) We show that recycling between models is possible (our best settings are able to successfully recycle $88.9%$ of prompts, producing a prompt that out-performs baselines) but significant performance headroom remains, requiring improved recycling techniques.
• Score: 23.6827516021001
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Parameter-efficient methods are able to use a single frozen pre-trained large language model (LLM) to perform many tasks by learning task-specific soft prompts that modulate model behavior when concatenated to the input text. However, these learned prompts are tightly coupled to a given frozen model -- if the model is updated, corresponding new prompts need to be obtained. In this work, we propose and investigate several approaches to "Prompt Recycling'" where a prompt trained on a source model is transformed to work with the new target model. Our methods do not rely on supervised pairs of prompts, task-specific data, or training updates with the target model, which would be just as costly as re-tuning prompts with the target model from scratch. We show that recycling between models is possible (our best settings are able to successfully recycle $88.9\%$ of prompts, producing a prompt that out-performs baselines), but significant performance headroom remains, requiring improved recycling techniques.

Related papers

• PromptBridge: Cross-Model Prompt Transfer for Large Language Models [28.175276092083735]
  Large language models (LLMs) underpin applications in code generation, mathematical reasoning, and agent-based reasoning.<n>Reusing a prompt engineered for one model on another often yields substantially worse performance than a prompt optimized for the target model.<n>We introduce PromptBridge, a training-free framework that preserves prompt effectiveness under model switches.
  arXiv  Detail & Related papers  (2025-12-01T08:55:45Z)
• TRPrompt: Bootstrapping Query-Aware Prompt Optimization from Textual Rewards [9.107586166322923]
  We introduce the Textual Reward Prompt framework (TRPrompt), which unifies approaches by incorporating textual feedback into training of the prompt model.<n>Our framework does not require prior dataset collection and is being iteratively improved with the feedback on the generated prompts.<n>When coupled with the capacity of an LLM to internalize the notion of what a "good" prompt is, the high-resolution signal provided by the textual rewards allows us to train a prompt model yielding state-of-the-art query-specific prompts.
  arXiv  Detail & Related papers  (2025-07-24T17:54:44Z)
• Sharpness-Aware Parameter Selection for Machine Unlearning [6.397490580631141]
  It often happens that some sensitive personal information, such as credit card numbers or passwords, are mistakenly incorporated in the training of machine learning models and need to be removed afterwards. There have been various machine unlearning techniques proposed in the literature to address this problem. Most of the proposed methods revolve around removing individual data samples from a trained model. While the existing methods for these tasks do the unlearning task by updating the whole set of model parameters or only the last layer of the model, we show that there are a subset of model parameters that have the largest contribution in the unlearning target features.
  arXiv  Detail & Related papers  (2025-04-08T19:41:07Z)
• Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration [90.41908331897639]
  Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data. We present a novel approach, ReverseGen, designed to automatically generate effective training samples.
  arXiv  Detail & Related papers  (2024-10-22T06:43:28Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• $\Delta$-Patching: A Framework for Rapid Adaptation of Pre-trained Convolutional Networks without Base Performance Loss [71.46601663956521]
  Models pre-trained on large-scale datasets are often fine-tuned to support newer tasks and datasets that arrive over time. We propose $Delta$-Patching for fine-tuning neural network models in an efficient manner, without the need to store model copies. Our experiments show that $Delta$-Networks outperform earlier model patching work while only requiring a fraction of parameters to be trained.
  arXiv  Detail & Related papers  (2023-03-26T16:39:44Z)
• Scalable Weight Reparametrization for Efficient Transfer Learning [10.265713480189486]
  Efficient transfer learning involves utilizing a pre-trained model trained on a larger dataset and repurposing it for downstream tasks. Previous works have led to an increase in updated parameters and task-specific modules, resulting in more computations, especially for tiny models. We suggest learning a policy network that can decide where to reparametrize the pre-trained model, while adhering to a given constraint for the number of updated parameters.
  arXiv  Detail & Related papers  (2023-02-26T23:19:11Z)
• Effective and Efficient Training for Sequential Recommendation using Recency Sampling [91.02268704681124]
  We propose a novel Recency-based Sampling of Sequences training objective. We show that the models enhanced with our method can achieve performances exceeding or very close to stateof-the-art BERT4Rec.
  arXiv  Detail & Related papers  (2022-07-06T13:06:31Z)
• Few-shot Prompting Towards Controllable Response Generation [49.479958672988566]
  We first explored the combination of prompting and reinforcement learning (RL) to steer models' generation without accessing any of the models' parameters. We apply multi-task learning to make the model learn to generalize to new tasks better. Experiment results show that our proposed method can successfully control several state-of-the-art (SOTA) dialogue models without accessing their parameters.
  arXiv  Detail & Related papers  (2022-06-08T14:48:06Z)
• Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation [87.98063273826702]
  We propose a memory imitation meta-learning (MemIML) method that enhances the model's reliance on support sets for task adaptation. A theoretical analysis is provided to prove the effectiveness of our method.
  arXiv  Detail & Related papers  (2022-03-22T12:41:55Z)
• Training Neural Networks with Fixed Sparse Masks [19.58969772430058]
  Recent work has shown that it is possible to update only a small subset of the model's parameters during training. We show that it is possible to induce a fixed sparse mask on the model's parameters that selects a subset to update over many iterations.
  arXiv  Detail & Related papers  (2021-11-18T18:06:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.