Self-Routed Tensor Adapters for Parameter-Efficient Universal Visual Adaptation

• URL: http://arxiv.org/abs/2608.16384v1
• Date: Mon, 17 Aug 2026 10:35:36 GMT
• Title: Self-Routed Tensor Adapters for Parameter-Efficient Universal Visual Adaptation
• Abstract summary: Self-Routed Adapters is a compact framework for multi-domain visual adaptation.<n>It produces a sample-specific adaptation matrix without an external gating network.<n>It achieves competitive or slightly stronger average accuracy than MoE-style PEFT baselines.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Universal visual representations require adaptation mechanisms that adapt across heterogeneous domains without fragmenting knowledge into domain-specific modules. Parameter-efficient fine-tuning adapts frozen visual foundation models efficiently, but standard low-rank adapters use a fixed subspace for all inputs, which can be restrictive when domains differ in style, background, and semantic context. MoE-based adapters improve specialization through multiple expert pathways, but often rely on external routers and large expert banks, adding parameters and separating routing from adaptation. We propose \textbf{Self-Routed Tensor Adapters}, a compact framework for multi-domain visual adaptation. SRTA projects each input into a low-rank space, computes routing weights from this representation using a learnable domain matrix, and uses these weights to blend slices of a shared Tucker core. This produces a sample-specific adaptation matrix without an external gating network, allowing shared visual factors to be reused while supporting domain-aware specialization. To strengthen pathway learning, we introduce a progressive depth-weighted routing objective that supervises routing decisions across adapter layers. Across five heterogeneous multi-domain visual classification benchmarks, SRTA achieves competitive or slightly stronger average accuracy than MoE-style PEFT baselines while using substantially fewer trainable parameters. At rank 64, SRTA uses 2.77M parameters in the 4-domain setting compared with 9.52M for MoLoRA, and 3.00M in the 6-domain setting compared with 14.31M. Overall, SRTA offers an effective accuracy-parameter trade-off for adapting visual foundation models toward universal multi-domain representations. \href{https://github.com/surajyadav-research/SRTA}{GitHub}

Related papers

• A Unified Adaptive Feature Composition Framework for Multi-Task Generalization in Wireless Foundation Models [19.520699593557357]
  We propose a unified adaptive feature composition framework for generalization in wireless foundation models (WFMs)<n>Instead of extracting only the final-layer output, this router treats the hidden states from different Transformer depths as a reusable pool of multi-level hidden features.<n>Experiments on four representative wireless tasks demonstrate that RAFC consistently outperforms conventional adaptation baselines.
  arXiv  Detail & Related papers  (2026-06-09T00:57:56Z)
• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  Vision-Language Models (VLMs) have broad potential in privacy-sensitive domains such as healthcare and finance.<n>We propose MoR, a federated alignment framework that combines GRPO with Mixture-of-Rewards for heterogeneousVLMs.<n>MoR consistently outperforms federated alignment baselines in generalization and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-05-05T07:02:50Z)
• Parameters as Experts: Adapting Vision Models with Dynamic Parameter Routing [41.836954056293614]
  AdaRoute is a new adapter-style method featuring a simple mixture-of-experts (MoE) architecture.<n> Dynamic weight matrices in AdaRoute modules facilitate low-rank adaptation in an input-dependent manner.<n>Experiments demonstrate the superiority of AdaRoute on diverse vision tasks.
  arXiv  Detail & Related papers  (2026-02-06T16:50:38Z)
• Beyond Weight Adaptation: Feature-Space Domain Injection for Cross-Modal Ship Re-Identification [3.6907522136316975]
  Cross-Modality Ship Re-Identification (CMS Re-ID) is critical for achieving all-day and all-weather maritime target tracking.<n>We explore the potential of Vision Foundation Models (VFMs) in bridging modality gaps.<n>We propose a novel PEFT strategy termed Domain Representation Injection (DRI)
  arXiv  Detail & Related papers  (2025-12-24T02:30:23Z)
• CATCH: A Modular Cross-domain Adaptive Template with Hook [2.869731339311564]
  CATCH is a plug-and-play framework for crossdomain adaptation of Visual Question Answering (VQA) models.<n>Our key idea is to decouple visual and linguistic adaptation by introducing two lightweight modules.<n>Results show that our framework achieves consistent performance gains without retraining the backbone model.
  arXiv  Detail & Related papers  (2025-10-30T15:10:02Z)
• ConvLoRA and AdaBN based Domain Adaptation via Self-Training [4.006331916849688]
  We propose Convolutional Low-Rank Adaptation (ConvLoRA) for multi-target domain adaptation. ConvLoRA freezes pre-trained model weights, adds trainable low-rank decomposition matrices to convolutional layers, and backpropagates the gradient. Our method has fewer trainable parameters and performs better or on-par with large independent fine-tuned networks.
  arXiv  Detail & Related papers  (2024-02-07T15:43:50Z)
• Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning [91.5113227694443]
  We propose a novel visual. sensuous-aware fine-Tuning (SPT) scheme. SPT allocates trainable parameters to task-specific important positions. Experiments on a wide range of downstream recognition tasks show that our SPT is complementary to the existing PEFT methods.
  arXiv  Detail & Related papers  (2023-03-15T12:34:24Z)
• AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models [127.04370753583261]
  Pretrained language models (PLMs) are trained on massive corpora, but often need to specialize to specific domains. A solution is to use a related-domain adapter for the novel domain at test time. We introduce AdapterSoup, an approach that performs weight-space averaging of adapters trained on different domains.
  arXiv  Detail & Related papers  (2023-02-14T13:09:23Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Pruning Self-attentions into Convolutional Layers in Single Path [89.55361659622305]
  Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. We propose Single-Path Vision Transformer pruning (SPViT) to efficiently and automatically compress the pre-trained ViTs. Our SPViT can trim 52.0% FLOPs for DeiT-B and get an impressive 0.6% top-1 accuracy gain simultaneously.
  arXiv  Detail & Related papers  (2021-11-23T11:35:54Z)
• Multi-path Neural Networks for On-device Multi-domain Visual Classification [55.281139434736254]
  This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths.
  arXiv  Detail & Related papers  (2020-10-10T05:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.