Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic

• URL: http://arxiv.org/abs/2407.07089v2
• Date: Wed, 29 Jan 2025 12:12:21 GMT
• Title: Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic
• Authors: Ruochen Jin, Bojian Hou, Jiancong Xiao, Weijie Su, Li Shen,
• Abstract summary: In recent years, task arithmetic has garnered increasing attention. This approach edits pre-trained models directly in weight space by combining the fine-tuned weights of various tasks into a unified model. Applying such a unified model to individual tasks can lead to interference from other tasks (lack of weight disentanglement)
• Score: 11.142414096809734
• License:
• Abstract: In recent years, task arithmetic has garnered increasing attention. This approach edits pre-trained models directly in weight space by combining the fine-tuned weights of various tasks into a unified model. Its efficiency and cost-effectiveness stem from its training-free combination, contrasting with traditional methods that require model training on large datasets for multiple tasks. However, applying such a unified model to individual tasks can lead to interference from other tasks (lack of weight disentanglement). To address this issue, Neural Tangent Kernel (NTK) linearization has been employed to leverage a "kernel behavior", facilitating weight disentanglement and mitigating adverse effects from unrelated tasks. Despite its benefits, NTK linearization presents drawbacks, including doubled training costs, as well as reduced performance of individual models. To tackle this problem, we propose a simple yet effective and efficient method that is to finetune the attention modules only in the Transformer. Our study reveals that the attention modules exhibit kernel behavior, and fine-tuning the attention modules only significantly improves weight disentanglement. To further understand how our method improves the weight disentanglement of task arithmetic, we present a comprehensive study of task arithmetic by differentiating the role of the representation module and task-specific module. In particular, we find that the representation module plays an important role in improving weight disentanglement whereas the task-specific modules such as the classification heads can degenerate the weight disentanglement performance. (The code is available at https://github.com/kyrie-23/task_arithmetic_tangent)

Related papers

• Multi-Task Model Merging via Adaptive Weight Disentanglement [69.7292615212444]
  We introduce an Adaptive Weight Disentanglement method for model merging. We successfully extract redundant vectors, and after their subtraction, the task vectors retain robust performance.
  arXiv  Detail & Related papers  (2024-11-27T20:08:55Z)
• Parameter Efficient Multi-task Model Fusion with Partial Linearization [97.23530944186078]
  We propose a novel method to improve multi-task fusion for parameter-efficient fine-tuning techniques. Our approach partially linearizes only the adapter modules and applies task arithmetic over the linearized adapters. We demonstrate that our partial linearization technique enables a more effective fusion of multiple tasks into a single model.
  arXiv  Detail & Related papers  (2023-10-07T08:55:54Z)
• Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs [71.56345106591789]
  It has been believed that weights in large language models (LLMs) contain significant redundancy. This paper presents a counter-argument: small-magnitude weights of pre-trained model weights encode vital knowledge essential for tackling difficult downstream tasks.
  arXiv  Detail & Related papers  (2023-09-29T22:55:06Z)
• Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models [96.9373147383119]
  We show that weight disentanglement is the crucial factor that makes task arithmetic effective. We show that fine-tuning models in their tangent space by linearizing them amplifies weight disentanglement. This leads to substantial performance improvements across task arithmetic benchmarks and diverse models.
  arXiv  Detail & Related papers  (2023-05-22T08:39:25Z)
• Scalable Weight Reparametrization for Efficient Transfer Learning [10.265713480189486]
  Efficient transfer learning involves utilizing a pre-trained model trained on a larger dataset and repurposing it for downstream tasks. Previous works have led to an increase in updated parameters and task-specific modules, resulting in more computations, especially for tiny models. We suggest learning a policy network that can decide where to reparametrize the pre-trained model, while adhering to a given constraint for the number of updated parameters.
  arXiv  Detail & Related papers  (2023-02-26T23:19:11Z)
• Editing Models with Task Arithmetic [69.97273155842966]
  Changing how pre-trained models behave is a common practice when developing machine learning systems. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition.
  arXiv  Detail & Related papers  (2022-12-08T05:50:53Z)
• Task Adaptive Parameter Sharing for Multi-Task Learning [114.80350786535952]
  Adaptive Task Adapting Sharing (TAPS) is a method for tuning a base model to a new task by adaptively modifying a small, task-specific subset of layers. Compared to other methods, TAPS retains high accuracy on downstream tasks while introducing few task-specific parameters. We evaluate our method on a suite of fine-tuning tasks and architectures (ResNet, DenseNet, ViT) and show that it achieves state-of-the-art performance while being simple to implement.
  arXiv  Detail & Related papers  (2022-03-30T23:16:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.