Task Addition and Weight Disentanglement in Closed-Vocabulary Models

• URL: http://arxiv.org/abs/2511.14569v1
• Date: Tue, 18 Nov 2025 15:12:21 GMT
• Title: Task Addition and Weight Disentanglement in Closed-Vocabulary Models
• Authors: Adam Hazimeh, Alessandro Favero, Pascal Frossard,
• Abstract summary: Task arithmetic has emerged as a promising method for editing pre-trained textitopen-vocabulary models.<n>In this paper, we study task addition in closed-vocabulary image classification models.<n>We find that pre-trained vision transformers can also be edited with task arithmetic.
• Score: 75.01322212415435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Task arithmetic has recently emerged as a promising method for editing pre-trained \textit{open-vocabulary} models, offering a cost-effective alternative to standard multi-task fine-tuning. However, despite the abundance of \textit{closed-vocabulary} models that are not pre-trained with language supervision, applying task arithmetic to these models remains unexplored. In this paper, we deploy and study task addition in closed-vocabulary image classification models. We consider different pre-training schemes and find that \textit{weight disentanglement} -- the property enabling task arithmetic -- is a general consequence of pre-training, as it appears in different pre-trained closed-vocabulary models. In fact, we find that pre-trained closed-vocabulary vision transformers can also be edited with task arithmetic, achieving high task addition performance and enabling the efficient deployment of multi-task models. Finally, we demonstrate that simple linear probing is a competitive baseline to task addition. Overall, our findings expand the applicability of task arithmetic to a broader class of pre-trained models and open the way for more efficient use of pre-trained models in diverse settings.

Related papers

• When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers [64.1656365676171]
  Task arithmetic refers to editing the pre-trained model by adding a weighted sum of task vectors.<n>This paper theoretically prove the effectiveness of task addition in simultaneously learning a set of irrelevant or irrelevant tasks.<n>We prove the proper selection for task arithmetic to achieve negation to out-of-domain tasks.
  arXiv  Detail & Related papers  (2025-04-15T08:04:39Z)
• Exploring Transferability for Randomized Smoothing [37.60675615521106]
  We propose a method for pretraining certifiably robust models. We find that surprisingly strong certified accuracy can be achieved even when finetuning on only clean images.
  arXiv  Detail & Related papers  (2023-12-14T15:08:27Z)
• Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models [96.9373147383119]
  We show that weight disentanglement is the crucial factor that makes task arithmetic effective. We show that fine-tuning models in their tangent space by linearizing them amplifies weight disentanglement. This leads to substantial performance improvements across task arithmetic benchmarks and diverse models.
  arXiv  Detail & Related papers  (2023-05-22T08:39:25Z)
• Editing Models with Task Arithmetic [69.97273155842966]
  Changing how pre-trained models behave is a common practice when developing machine learning systems. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition.
  arXiv  Detail & Related papers  (2022-12-08T05:50:53Z)
• Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization [94.4409074435894]
  We propose a novel and effective fine-tuning framework, named Layerwise Noise Stability Regularization (LNSR) Specifically, we propose to inject the standard Gaussian noise and regularize hidden representations of the fine-tuned model. We demonstrate the advantages of the proposed method over other state-of-the-art algorithms including L2-SP, Mixout and SMART.
  arXiv  Detail & Related papers  (2022-06-12T04:42:49Z)
• Bridging Pre-trained Models and Downstream Tasks for Source Code Understanding [13.65914588243695]
  We propose an approach to bridge pre-trained models and code-related tasks. We exploit semantic-preserving transformation to enrich downstream data diversity. We introduce curriculum learning to organize the transformed data in an easy-to-hard manner to fine-tune existing pre-trained models.
  arXiv  Detail & Related papers  (2021-12-04T07:21:28Z)
• Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates [52.164757178369804]
  Recent advances in transfer learning for natural language processing in conjunction with active learning open the possibility to significantly reduce the necessary annotation budget. We conduct an empirical study of various Bayesian uncertainty estimation methods and Monte Carlo dropout options for deep pre-trained models in the active learning framework. We also demonstrate that to acquire instances during active learning, a full-size Transformer can be substituted with a distilled version, which yields better computational performance.
  arXiv  Detail & Related papers  (2021-01-20T13:59:25Z)
• Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation [4.538165276831437]
  We introduce a simple and efficient method, called Auxiliary Tuning, for adapting a pre-trained Language Model to a novel task. We demonstrate this approach on the task of conditional text generation.
  arXiv  Detail & Related papers  (2020-06-30T14:00:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.