Training-free Motion Factorization for Compositional Video Generation

• URL: http://arxiv.org/abs/2603.09104v1
• Date: Tue, 10 Mar 2026 02:27:48 GMT
• Title: Training-free Motion Factorization for Compositional Video Generation
• Abstract summary: We propose a motion factorization framework that decomposes complex motion into three primary categories: motionlessness, rigid motion, and non-rigid motion.<n>Our framework achieves impressive performance in motion synthesis on real-world benchmarks.
• Score: 57.819757612370374
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compositional video generation aims to synthesize multiple instances with diverse appearance and motion, which is widely applicable in real-world scenarios. However, current approaches mainly focus on binding semantics, neglecting to understand diverse motion categories specified in prompts. In this paper, we propose a motion factorization framework that decomposes complex motion into three primary categories: motionlessness, rigid motion, and non-rigid motion. Specifically, our framework follows a planning before generation paradigm. (1) During planning, we reason about motion laws on the motion graph to obtain frame-wise changes in the shape and position of each instance. This alleviates semantic ambiguities in the user prompt by organizing it into a structured representation of instances and their interactions. (2) During generation, we modulate the synthesis of distinct motion categories in a disentangled manner. Conditioned on the motion cues, guidance branches stabilize appearance in motionless regions, preserve rigid-body geometry, and regularize local non-rigid deformations. Crucially, our two modules are model-agnostic, which can be seamlessly incorporated into various diffusion model architectures. Extensive experiments demonstrate that our framework achieves impressive performance in motion synthesis on real-world benchmarks. Our code will be released soon.

Related papers

• Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations [32.70813770106954]
  Video motion transfer aims to animate a target object using dynamics from a reference video.<n>Existing formulations largely rely on fixed structural correspondence.<n>We introduce Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence.
  arXiv  Detail & Related papers  (2026-08-03T02:59:54Z)
• Neural Motion Blending Across Arbitrary Character Topologies [7.503125656408304]
  We present a novel framework for motion blending across heterogeneous skeletons.<n>The proposed architecture combines a semantic encoder, which extracts per-frame latent representations of the motion state, with a diffusion-based decoder.<n>We train and evaluate the method on the Truebones Zoo dataset using motions defined on both same and distinct skeleton topologies.
  arXiv  Detail & Related papers  (2026-07-11T15:46:13Z)
• IAM: Identity-Aware Human Motion and Shape Joint Generation [14.037850965399384]
  We propose an identity-aware motion generation framework that explicitly models the relationship between body morphology and motion dynamics.<n>We introduce a joint motion-shape generation paradigm that simultaneously synthesizes motion sequences and body shape parameters.<n>Experiments on motion capture datasets and large-scale in-the-wild videos demonstrate improved motion realism and motion-identity consistency.
  arXiv  Detail & Related papers  (2026-04-28T03:15:12Z)
• CoMo: Compositional Motion Customization for Text-to-Video Generation [40.446146411270156]
  CoMo is a novel framework for $textbfcompositional motion customization$ in text-to-video generation.<n>It addresses the challenges of motion-merge entanglement and ineffective multi-motion blending.<n>CoMo achieves state-of-the-art performance, significantly advancing the capabilities of controllable video generation.
  arXiv  Detail & Related papers  (2025-10-27T04:57:09Z)
• SynMotion: Semantic-Visual Adaptation for Motion Customized Video Generation [56.90807453045657]
  SynMotion is a motion-customized video generation model that jointly leverages semantic guidance and visual adaptation.<n>At the semantic level, we introduce the dual-em semantic comprehension mechanism which disentangles subject and motion representations.<n>At the visual level, we integrate efficient motion adapters into a pre-trained video generation model to enhance motion fidelity and temporal coherence.
  arXiv  Detail & Related papers  (2025-06-30T10:09:32Z)
• KinMo: Kinematic-aware Human Motion Understanding and Generation [6.962697597686156]
  Current human motion synthesis frameworks rely on global action descriptions.<n>A single coarse description, such as run, fails to capture details such as variations in speed, limb positioning, and kinematic dynamics.<n>We introduce KinMo, a unified framework built on a hierarchical describable motion representation.
  arXiv  Detail & Related papers  (2024-11-23T06:50:11Z)
• Animate Your Motion: Turning Still Images into Dynamic Videos [58.63109848837741]
  We introduce Scene and Motion Conditional Diffusion (SMCD), a novel methodology for managing multimodal inputs. SMCD incorporates a recognized motion conditioning module and investigates various approaches to integrate scene conditions. Our design significantly enhances video quality, motion precision, and semantic coherence.
  arXiv  Detail & Related papers  (2024-03-15T10:36:24Z)
• DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion [70.33381660741861]
  We present DiverseMotion, a new approach for synthesizing high-quality human motions conditioned on textual descriptions. We show that our DiverseMotion achieves the state-of-the-art motion quality and competitive motion diversity.
  arXiv  Detail & Related papers  (2023-09-04T05:43:48Z)
• MoDi: Unconditional Motion Synthesis from Diverse Data [51.676055380546494]
  We present MoDi, an unconditional generative model that synthesizes diverse motions. Our model is trained in a completely unsupervised setting from a diverse, unstructured and unlabeled motion dataset. We show that despite the lack of any structure in the dataset, the latent space can be semantically clustered.
  arXiv  Detail & Related papers  (2022-06-16T09:06:25Z)
• Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis [117.15586710830489]
  We focus on the problem of synthesizing diverse scene-aware human motions under the guidance of target action sequences. Based on this factorized scheme, a hierarchical framework is proposed, with each sub-module responsible for modeling one aspect. Experiment results show that the proposed framework remarkably outperforms previous methods in terms of diversity and naturalness.
  arXiv  Detail & Related papers  (2022-05-25T18:20:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.