Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization

• URL: http://arxiv.org/abs/2606.26668v1
• Date: Thu, 25 Jun 2026 07:01:05 GMT
• Title: Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization
• Abstract summary: Video customization based on Text-to-Video (T2V) models aims to learn specific features from reference data to generate controllable videos.<n>We systematically define the task of multi-concept video customization, which requires the joint control of content, style, and motion.<n>We propose Disco-LoRA, a unified framework designed to tackle this problem by disentangling and flexibly recombining different concepts in two stages.
• Score: 24.028888270203215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video customization based on Text-to-Video (T2V) models aims to learn specific features from reference data to generate controllable videos. While significant strides have been made in image stylization and video motion customization, simultaneously controlling multiple concepts, such as content, style, and motion, remains a major challenge. In this work, we systematically define the task of multi-concept video customization, which requires the joint control of content, style, and motion. To facilitate research in this area, we construct a comprehensive benchmark and propose Disco-LoRA, a unified framework designed to tackle this problem by disentangling and flexibly recombining different concepts in two stages: (1) We decompose the objective into two sub-tasks: Content-Style and Content-Motion. Each sub-task is addressed using our Iterative Dual-LoRA Disentanglement Framework, which effectively disentangles distinct concepts within the data. (2) We identify layer-wise weight trends as crucial for LoRA identity, while weight magnitudes dictate composability. To harmonize these scales, we propose a Z-score-based statistical regularization that aligns weight distributions, preserving layer-wise trends while minimizing interference between different LoRAs. Extensive experiments show that Disco-LoRA excels in multi-concept video customization, effectively preserving appearance, style, and motion for controllable text-to-video generation.

Related papers

• DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning [82.28800081483224]
  We present DreamVideo- Omni, a unified framework enabling multisubject customization with omni-motion control.<n>To resolve multi-subject ambiguity, we introduce group and role embeddings to explicitly anchor motion signals to specific identities.<n>This provides motion-aware identity rewards in the latent space, prioritizing identity preservation aligned with human preferences.
  arXiv  Detail & Related papers  (2026-03-12T17:59:12Z)
• Video2LoRA: Unified Semantic-Controlled Video Generation via Per-Reference-Video LoRA [17.6011255931793]
  We propose Video2LoRA, a scalable and generalizable framework for semantic-controlled video generation.<n>Video2LoRA employs a lightweight hypernetwork to predict personalized LoRA weights for each semantic input.<n>The model generates videos consistent with the reference semantics while preserving key style and content variations.
  arXiv  Detail & Related papers  (2026-03-09T10:40:03Z)
• UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework [54.337290937468175]
  We propose UniMo, an autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework.<n>We show that our method simultaneously generates corresponding videos and motions while performing accurate motion capture.
  arXiv  Detail & Related papers  (2025-12-03T16:03:18Z)
• FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning [65.42201665046505]
  Current video understanding models rely on fixed frame sampling strategies, processing predetermined visual inputs regardless of the specific reasoning requirements of each question.<n>This static approach limits their ability to adaptively gather visual evidence, leading to suboptimal performance on tasks that require broad temporal coverage or fine-grained spatial detail.<n>We introduce FrameMind, an end-to-end framework trained with reinforcement learning that enables models to dynamically request visual information during reasoning through Frame-Interleaved Chain-of-Thought (FiCOT)<n>Unlike traditional approaches, FrameMind operates in multiple turns where the model alternates between textual reasoning and active visual perception, using tools to extract
  arXiv  Detail & Related papers  (2025-09-28T17:59:43Z)
• Zero-Shot Dynamic Concept Personalization with Grid-Based LoRA [84.89284738178932]
  We introduce a zero-shot framework for dynamic concept personalization in text-to-video models.<n>Our method leverages structured 2x2 video grids that spatially organize input and output pairs.<n>A dedicated Grid Fill module completes partially observed layouts, producing temporally coherent and identity preserving outputs.
  arXiv  Detail & Related papers  (2025-07-23T22:09:38Z)
• Leveraging Modality Tags for Enhanced Cross-Modal Video Retrieval [26.40393400497247]
  Video retrieval requires aligning visual content with corresponding natural language descriptions.<n>In this paper, we introduce Modality Auxiliary Concepts for Video Retrieval (MAC-VR)<n>We propose to align modalities in a latent space, along with learning and aligning auxiliary latent concepts.
  arXiv  Detail & Related papers  (2025-04-02T10:56:01Z)
• JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation [13.168628936598367]
  JointTuner is a framework that enables joint optimization of appearance and motion components.<n>AiT Loss disrupts the flow of appearance-related components, guiding the model to focus exclusively on motion learning.<n>JointTuner is compatible with both UNet-based models and Diffusion Transformer-based models.
  arXiv  Detail & Related papers  (2025-03-31T11:04:07Z)
• Dynamic Concepts Personalization from Single Videos [92.62863918003575]
  We introduce Set-and-Sequence, a novel framework for personalizing generative video models with dynamic concepts.<n>Our approach imposes a-temporal weight space within an architecture that does not explicitly separate spatial and temporal features.<n>Our framework embeds dynamic concepts into the video model's output domain, enabling unprecedented editability and compositionality.
  arXiv  Detail & Related papers  (2025-02-20T18:53:39Z)
• DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control [42.506988751934685]
  We present DreamVideo-2, a zero-shot video customization framework capable of generating videos with a specific subject and motion trajectory. Specifically, we introduce reference attention, which leverages the model's inherent capabilities for subject learning. We devise a mask-guided motion module to achieve precise motion control by fully utilizing the robust motion signal of box masks.
  arXiv  Detail & Related papers  (2024-10-17T17:52:57Z)
• Implicit Style-Content Separation using B-LoRA [61.664293840163865]
  We introduce B-LoRA, a method that implicitly separate the style and content components of a single image. By analyzing the architecture of SDXL combined with LoRA, we find that jointly learning the LoRA weights of two specific blocks achieves style-content separation.
  arXiv  Detail & Related papers  (2024-03-21T17:20:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.