DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

• URL: http://arxiv.org/abs/2607.17467v1
• Date: Mon, 20 Jul 2026 01:45:37 GMT
• Title: DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation
• Abstract summary: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples.<n>We introduce a new FSTT-DA framework that integrates LoRA fine-tuning with model merging.<n>Our results demonstrate state-of-the-art performance across various domain adaptation datasets.
• Score: 13.200133063912624
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples. This setting is more realistic than typical domain adaptation setups, which assume access to target data during source training. However, prior FSTT-DA approaches fail to effectively leverage source domain-specific knowledge, relying on shallow batch normalization updates, prompt-based methods that treat the model as a black box, or ensembling strategies that do not capture cross-domain relationships. To address these limitations, we introduce a new FSTT-DA framework that integrates LoRA fine-tuning with model merging. In our approach, separate LoRA modules are fine-tuned on CLIP's vision encoder for each source domain. Since LoRA modifies only a small fraction of the model's parameters, it retains the base model's generalized knowledge while internally learning domain-specific features. To adapt the learned knowledge to a specific target domain, we propose a hypernetwork trained via meta-learning that generates per-column merging factors to combine LoRA modules. Given a small batch of target images, the hypernetwork produces merging weights that fuse source LoRA modules into a single adapted representation. Our results demonstrate state-of-the-art performance across various domain adaptation datasets. Our code is publicly available at https://github.com/nahbois4321/DA-MergeLoRA.

Related papers

• WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Merging [24.949880939628386]
  We introduce WeaveRec, which cross-trains multiple LoRA modules with source and target domain data in a weaving fashion.<n>We provide a theoretical guarantee that WeaveRec can reduce the upper bound of the expected error in the target domain.
  arXiv  Detail & Related papers  (2025-10-30T14:37:15Z)
• HiLoRA: Adaptive Hierarchical LoRA Routing for Training-Free Domain Generalization [39.23407996213986]
  Low-Rank Adaptation (LoRA) has emerged as a widely used technique for adapting large language models to new domains.<n>Existing methods often rely on explicit task labels or additional training, which are impractical for deployment.<n>We propose textttHiLoRA, a training-free framework that performs adaptive hierarchical routing over LoRA pools.
  arXiv  Detail & Related papers  (2025-10-14T08:19:13Z)
• RegCL: Continual Adaptation of Segment Anything Model via Model Merging [6.868344361490698]
  This paper proposes RegCL, a novel non-replay continual learning framework for efficient multi-domain knowledge integration.<n>RegCL incorporates the model merging algorithm into the continual learning paradigm by merging the parameters of SAM's adaptation modules.<n> Experimental results demonstrate that RegCL achieves favorable continual learning performance across multiple downstream datasets.
  arXiv  Detail & Related papers  (2025-07-16T14:51:37Z)
• Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation [66.72195610471624]
  Cross-Domain Sequential Recommendation aims to mine and transfer users' sequential preferences across different domains. We propose a novel framework named URLLM, which aims to improve the CDSR performance by exploring the User Retrieval approach.
  arXiv  Detail & Related papers  (2024-06-05T09:19:54Z)
• Virtual Classification: Modulating Domain-Specific Knowledge for Multidomain Crowd Counting [67.38137379297717]
  Multidomain crowd counting aims to learn a general model for multiple diverse datasets. Deep networks prefer modeling distributions of the dominant domains instead of all domains, which is known as domain bias. We propose a Modulating Domain-specific Knowledge Network (MDKNet) to handle the domain bias issue in multidomain crowd counting.
  arXiv  Detail & Related papers  (2024-02-06T06:49:04Z)
• FedRA: A Random Allocation Strategy for Federated Tuning to Unleash the Power of Heterogeneous Clients [50.13097183691517]
  In real-world federated scenarios, there often exist a multitude of heterogeneous clients with varying computation and communication resources. We propose a novel federated tuning algorithm, FedRA. In each communication round, FedRA randomly generates an allocation matrix. It reorganizes a small number of layers from the original model based on the allocation matrix and fine-tunes using adapters.
  arXiv  Detail & Related papers  (2023-11-19T04:43:16Z)
• Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval [55.122020263319634]
  Video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query. In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain but the domain of interest only contains unannotated datasets. We propose a novel Multi-Modal Cross-Domain Alignment network to transfer the annotation knowledge from the source domain to the target domain.
  arXiv  Detail & Related papers  (2022-09-23T12:58:20Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation [102.67010690592011]
  Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model. This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
  arXiv  Detail & Related papers  (2020-02-20T03:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.