FreeFuse: Multi-Subject LoRA Fusion via Auto Masking at Test Time

• URL: http://arxiv.org/abs/2510.23515v1
• Date: Mon, 27 Oct 2025 16:54:08 GMT
• Title: FreeFuse: Multi-Subject LoRA Fusion via Auto Masking at Test Time
• Authors: Yaoli Liu, Yao-Xiang Ding, Kun Zhou,
• Abstract summary: FreeFuse is a training-free approach for multi-subject text-to-image generation through automatic fusion of multiple subject LoRAs.<n>It demonstrates superior practicality and efficiency as it requires no additional training, no modification to LoRAs, no auxiliary models, and no user-defined prompt templates or region specifications.
• Score: 22.751943468067633
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes FreeFuse, a novel training-free approach for multi-subject text-to-image generation through automatic fusion of multiple subject LoRAs. In contrast to existing methods that either focus on pre-inference LoRA weight merging or rely on segmentation models and complex techniques like noise blending to isolate LoRA outputs, our key insight is that context-aware dynamic subject masks can be automatically derived from cross-attention layer weights. Mathematical analysis shows that directly applying these masks to LoRA outputs during inference well approximates the case where the subject LoRA is integrated into the diffusion model and used individually for the masked region. FreeFuse demonstrates superior practicality and efficiency as it requires no additional training, no modification to LoRAs, no auxiliary models, and no user-defined prompt templates or region specifications. Alternatively, it only requires users to provide the LoRA activation words for seamless integration into standard workflows. Extensive experiments validate that FreeFuse outperforms existing approaches in both generation quality and usability under the multi-subject generation tasks. The project page is at https://future-item.github.io/FreeFuse/

Related papers

• When LoRA Betrays: Backdooring Text-to-Image Models by Masquerading as Benign Adapters [10.859491015719088]
  Low-Rank Adaptation (LoRA) has emerged as a leading technique for efficiently fine-tuning text-to-image diffusion models.<n>MasqLoRA is the first systematic attack framework that leverages an independent LoRA module as the attack vehicle.<n>MasqLoRA achieves a high attack success rate of 99.8%.
  arXiv  Detail & Related papers  (2026-02-25T14:56:51Z)
• LoRAtorio: An intrinsic approach to LoRA Skill Composition [11.429106388558925]
  Low-Rank Adaptation (LoRA) has become a widely adopted technique in text-to-image diffusion models.<n>Existing approaches struggle to effectively compose multiple LoRA adapters.<n>We present LoRAtorio, a novel train-free framework for multi-LoRA composition.
  arXiv  Detail & Related papers  (2025-08-15T17:52:56Z)
• LoRA-Gen: Specializing Large Language Model via Online LoRA Generation [68.01864057372067]
  We propose the LoRA-Gen framework to generate LoRA parameters for edge-side models based on task descriptions.<n>We merge the LoRA parameters into the edge-side model to achieve flexible specialization.<n>Our method facilitates knowledge transfer between models while significantly improving the inference efficiency of the specialized model.
  arXiv  Detail & Related papers  (2025-06-13T10:11:01Z)
• A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models [22.457766373989365]
  Low-Rank Adapters (LoRAs) have been substantially adopted across various fields, including instruction tuning and domain adaptation.<n>To address the limited expressive capacity of LoRA, the Mixture-of-Expert (MoE) has been introduced for incorporating multiple LoRA adapters.<n>We propose a new training strategy for MoE-LoRA, to stabilize and boost its feature learning procedure by multi-space projections.
  arXiv  Detail & Related papers  (2025-02-20T05:58:53Z)
• In-Context Meta LoRA Generation [61.690065588534296]
  Low-rank Adaptation (LoRA) has demonstrated remarkable capabilities for task specific fine-tuning.<n>We propose In-Context Meta LoRA (ICM-LoRA), a novel approach that efficiently achieves task-specific customization of large language models.<n>ICM-LoRA enables more accurate LoRA parameter reconstruction than current parameter reconstruction methods.
  arXiv  Detail & Related papers  (2025-01-29T13:12:01Z)
• Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning [53.053604713064544]
  Low-Rank Adaptation (LoRA) is widely used for adapting large language models (LLMs) to specific domains due to its efficiency and modularity.<n>Recent works adopt Mixture of Experts (MoE) by treating each LoRA module as an expert, thereby mitigating task interference through multiple specialized LoRA modules.<n>While effective, these methods often isolate knowledge within individual tasks, failing to fully exploit the shared knowledge across related tasks.<n>We propose Single-ranked Mixture of Experts LoRA (textbfSMoRA), which embeds MoE into LoRA by textittreating each rank as an
  arXiv  Detail & Related papers  (2025-01-25T06:56:39Z)
• AutoLoRA: AutoGuidance Meets Low-Rank Adaptation for Diffusion Models [0.9514837871243403]
  Low-rank adaptation (LoRA) is a fine-tuning technique that can be applied to conditional generative diffusion models. We introduce AutoLoRA, a novel guidance technique for diffusion models fine-tuned with the LoRA approach.
  arXiv  Detail & Related papers  (2024-10-04T21:57:11Z)
• Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning [57.36978335727009]
  Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs) In this paper, we propose a framework that adaptively retrieves and composes multiple LoRAs based on input prompts.
  arXiv  Detail & Related papers  (2024-06-24T05:24:41Z)
• Mixture-of-Subspaces in Low-Rank Adaptation [19.364393031148236]
  We introduce a subspace-inspired Low-Rank Adaptation (LoRA) method, which is computationally efficient, easy to implement, and readily applicable to large language, multimodal, and diffusion models.<n>To be more flexible, we jointly learn the mixer with the original LoRA weights, and term the method Mixture-of-Subspaces LoRA.<n>MoSLoRA consistently outperforms LoRA on tasks in different modalities, including commonsense reasoning, visual instruction tuning, and subject-driven text-to-image generation.
  arXiv  Detail & Related papers  (2024-06-16T14:19:49Z)
• MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts [3.6301530893494127]
  MixLoRA is an approach to construct a resource-efficient sparse MoE model based on LoRA. Our evaluations show that MixLoRA improves about 9% accuracy compared to state-of-the-art PEFT methods in multi-task learning scenarios.
  arXiv  Detail & Related papers  (2024-04-22T02:15:52Z)
• Mixture of LoRA Experts [87.50120181861362]
  This paper introduces the Mixture of LoRA Experts (MoLE) approach, which harnesses hierarchical control and unfettered branch selection. The MoLE approach achieves superior LoRA fusion performance in comparison to direct arithmetic merging.
  arXiv  Detail & Related papers  (2024-04-21T11:59:53Z)
• Continual Forgetting for Pre-trained Vision Models [70.51165239179052]
  In real-world scenarios, selective information is expected to be continuously removed from a pre-trained model. We propose Group Sparse LoRA (GS-LoRA) for efficient and effective deleting. We conduct extensive experiments on face recognition, object detection and image classification and demonstrate that GS-LoRA manages to forget specific classes with minimal impact on other classes.
  arXiv  Detail & Related papers  (2024-03-18T07:33:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.