AdaRing: Towards Ultra-Light Vision-Language Adaptation via Cross-Layer Tensor Ring Decomposition

• URL: http://arxiv.org/abs/2508.11870v2
• Date: Tue, 19 Aug 2025 21:43:34 GMT
• Title: AdaRing: Towards Ultra-Light Vision-Language Adaptation via Cross-Layer Tensor Ring Decomposition
• Authors: Ying Huang, Yuanbin Man, Wenqi Jia, Zhengzhong Tu, Junzhou Huang, Miao Yin,
• Abstract summary: We propose a vision-language fine-tuning framework based on cross-layer tensor ring decomposition (TRD) with the integration and collaboration of diverse adapters, called AdaRing.<n>Our experiments show that the proposed AdaRing achieves the state-of-the-art performance while reducing average training parameters by 90%.
• Score: 41.654675205772485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adapter-based fine-tuning has gained remarkable attention in adapting large pre-trained vision language models (VLMs) for a wide range of downstream tasks efficiently. In this paradigm, only the inserted adapters are fine-tuned, without the need for training the original VLM backbone. Existing works scale adapters by integrating them into every layer of VLMs to increase the capacity of adapters. However, these methods face two primary limitations: 1) limited compression rate due to ignoring cross-layer redundancy, and 2) limited representational capacity across homogeneous adapters. In this paper, we propose a novel vision-language fine-tuning framework based on cross-layer tensor ring decomposition (TRD) with the integration and collaboration of diverse adapters, called AdaRing, achieving ultra-light parameter-efficient adaptation of VLMs on various tasks. To remove the high redundancy that exists among adapters across layers, we exploit the tensor-level low-rankness to formulate adapters as layer-shared tensor cores and layer-specific slices. Moreover, guided by generalization-aware fine-tuning, diverse rank-driven adapters cooperate to handle tasks that require different representations. Our experiments show that the proposed AdaRing achieves the state-of-the-art performance while reducing average training parameters by 90%.

Related papers

• Come Together, But Not Right Now: A Progressive Strategy to Boost Low-Rank Adaptation [21.137278840000366]
  Low-rank adaptation (LoRA) has emerged as a leading parameter-efficient fine-tuning technique for adapting large foundation models.<n>We propose CoTo pruning, a progressive training strategy that gradually increases adapters' activation probability over the course of fine-tuning.
  arXiv  Detail & Related papers  (2025-06-06T03:33:06Z)
• MoSA: Mixture of Sparse Adapters for Visual Efficient Tuning [20.68925288222065]
  Mixture of Sparse Adapters, or MoSA, is a novel Adapter Tuning method. MoSA can achieve significantly better performance than standard without any additional computational storage overhead. MoSA consistently outperforms other Adapter Tuning methods as well as other baselines by a large margin.
  arXiv  Detail & Related papers  (2023-12-05T17:50:55Z)
• Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing [8.88477151877883]
  High-capacity pre-trained models have revolutionized problem-solving in computer vision. We propose a novel Adapter Re-Composing (ARC) strategy that addresses efficient pre-trained model adaptation. Our approach considers the reusability of adaptation parameters and introduces a parameter-sharing scheme.
  arXiv  Detail & Related papers  (2023-10-10T01:04:15Z)
• Selective Feature Adapter for Dense Vision Transformers [30.409313135985528]
  selective feature adapter (SFA) achieves comparable or better performance than fully fine-tuned models across various dense tasks. SFA consists of external adapters and internal adapters which are sequentially operated over a transformer model. Experiments show that the dual adapter module, a.k.a SFA, is essential to achieve the best trade-off on dense vision tasks.
  arXiv  Detail & Related papers  (2023-10-03T07:17:58Z)
• MerA: Merging Pretrained Adapters For Few-Shot Learning [71.44422347502409]
  We propose textbftextttMerging Pretrained Adapters (MerA) that efficiently incorporates pretrained adapters to a single model through model fusion. Experiments on two PLMs demonstrate that MerA substantial improvements compared to both single adapters and AdapterFusion.
  arXiv  Detail & Related papers  (2023-08-30T12:10:17Z)
• Vision Transformer Adapters for Generalizable Multitask Learning [61.79647180647685]
  We introduce the first multitasking vision transformer adapters that learn generalizable task affinities. Our adapters can simultaneously solve multiple dense vision tasks in a parameter-efficient manner. In contrast to concurrent methods, we do not require retraining or fine-tuning whenever a new task or domain is added.
  arXiv  Detail & Related papers  (2023-08-23T18:40:48Z)
• AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models [119.7093605087114]
  Fine-tuning large-scale pre-trained language models to downstream tasks require updating hundreds of millions of parameters. This not only increases the serving cost to store a large copy of the model weights for every task, but also exhibits instability during few-shot task adaptation. We introduce a new mechanism to improve adapter capacity without increasing parameters or computational cost by two key techniques.
  arXiv  Detail & Related papers  (2022-05-24T23:41:22Z)
• Adaptable Adapters [74.65986170056945]
  State-of-the-art pretrained NLP models contain a hundred million to trillion parameters. Adaptable adapters contain different activation functions for different layers and different input data. We show that adaptable adapters achieve on-par performances with the standard adapter architecture.
  arXiv  Detail & Related papers  (2022-05-03T14:59:27Z)
• AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks [55.705355299065474]
  Transformer-based pre-trained models with millions of parameters require large storage. Recent approaches tackle this shortcoming by training adapters, but these approaches still require a relatively large number of parameters. In this study, AdapterBias, a surprisingly simple yet effective adapter architecture, is proposed.
  arXiv  Detail & Related papers  (2022-04-30T16:49:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.