Guided Motion Diffusion for Controllable Human Motion Synthesis

• URL: http://arxiv.org/abs/2305.12577v3
• Date: Sun, 29 Oct 2023 19:27:38 GMT
• Title: Guided Motion Diffusion for Controllable Human Motion Synthesis
• Abstract summary: We propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that manipulates motion representation to enhance the coherency between spatial information and local poses. Our experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation.
• Score: 18.660523853430497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Denoising diffusion models have shown great promise in human motion synthesis conditioned on natural language descriptions. However, integrating spatial constraints, such as pre-defined motion trajectories and obstacles, remains a challenge despite being essential for bridging the gap between isolated human motion and its surrounding environment. To address this issue, we propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that manipulates motion representation to enhance the coherency between spatial information and local poses. Together with a new imputation formulation, the generated motion can reliably conform to spatial constraints such as global motion trajectories. Furthermore, given sparse spatial constraints (e.g. sparse keyframes), we introduce a new dense guidance approach to turn a sparse signal, which is susceptible to being ignored during the reverse steps, into denser signals to guide the generated motion to the given constraints. Our extensive experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation while allowing control of the synthesized motions with spatial constraints.

Related papers

• DeSeG: Decoupling Semantic Intent and Geometric Constraints for Physically Plausible Human-Scene Interaction [50.06148080975928]
  We propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints.<n>DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.
  arXiv  Detail & Related papers  (2026-07-07T03:21:08Z)
• Beyond MoCap: Scaling Motion Tokenizers with Synthetic Human Motion for Generative Modeling [32.83313595986613]
  We propose a framework for expanding the motion representation space by leveraging large-scale synthetic human motion.<n>Unlike conventional tokenizers trained on narrow data distributions, our approach jointly scales both the training distribution and the discrete codebook.<n>We demonstrate that training with synthetic motion substantially improves the coverage and compositionality of the learned motion vocabulary.
  arXiv  Detail & Related papers  (2026-06-25T20:50:31Z)
• MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation [66.66098171359995]
  MotionMERGE is a unified framework that bridges the granularity gap in motion-language models.<n>First, we pioneer the study of fine-grained language-guided motion control, including detailed understanding and localized editing.<n>Second, we design ReasoningAware Granularity-Synergy pre-training, a novel strategy that employs joint supervision for cross-granularity alignment.<n>Third, we curate MotionFineEdit, a large-scale dataset with the first finegrained-temporal corrective instructions and motion-grounded CoT annotations.
  arXiv  Detail & Related papers  (2026-05-18T18:00:04Z)
• Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation [21.368303527876908]
  Trajectory-controlled human motion generation aims to synthesize realistic human motions conditioned on both textual descriptions and spatial trajectories.<n>Existing methods suffer from two critical limitations: first, the conflict between text and trajectory conditions disrupts the denoising process, resulting in compromised motion quality or inaccurate trajectory following.<n>We propose CMC, a decoupled framework that effectively coordinates text and trajectory conditions through a divide-and-conquer strategy.
  arXiv  Detail & Related papers  (2026-05-13T16:09:04Z)
• Hypergraph-State Collaborative Reasoning for Multi-Object Tracking [63.32950991964095]
  We propose a collaborative reasoning framework that enhances motion estimation through joint inference among multiple correlated objects.<n>By allowing objects with similar motion states to mutually constrain and refine each other, our framework stabilizes noisy trajectories and infers plausible motion continuity even when target is occluded.
  arXiv  Detail & Related papers  (2026-04-14T12:37:39Z)
• MotionCFG: Boosting Motion Dynamics via Stochastic Concept Perturbation [60.48914865049489]
  MotionCFG is a framework that enhances motion dynamics by contrasting a target concept with its noise-blured counterparts.<n>Unlike explicit negations, this approach facilitates implicit hard negative guidance without shifting the global semantic identity.<n> MotionCFG consistently improves motion dynamics across state-of-the-art T2V frameworks.
  arXiv  Detail & Related papers  (2026-03-14T18:39:37Z)
• IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation [58.297199313494]
  Implicit methods capture motion semantics directly from driving video, but suffer from identity leakage and entanglement between motion and appearance.<n>We propose a novel implicit motion representation that compresses per-frame motion into compact 1D motion tokens.<n>Our methodology employs a three-stage training strategy to enhance the training efficiency and ensure high fidelity.
  arXiv  Detail & Related papers  (2026-02-07T11:17:20Z)
• PMG: Progressive Motion Generation via Sparse Anchor Postures Curriculum Learning [5.247557449370603]
  ProMoGen is a novel framework that integrates trajectory guidance with sparse anchor motion control. ProMoGen supports both dual and single control paradigms within a unified training process. Our approach seamlessly integrates personalized motion with structured guidance, significantly outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-23T13:51:42Z)
• Leader and Follower: Interactive Motion Generation under Trajectory Constraints [42.90788442575116]
  This paper explores the motion range refinement process in interactive motion generation. It proposes a training-free approach, integrating a Pace Controller and a Kinematic Synchronization Adapter. Experimental results show that the proposed approach, by better leveraging trajectory information, outperforms existing methods in both realism and accuracy.
  arXiv  Detail & Related papers  (2025-02-17T08:52:45Z)
• Motion-Aware Generative Frame Interpolation [23.380470636851022]
  Flow-based frame methods ensure motion stability through estimated intermediate flow but often introduce severe artifacts in complex motion regions. Recent generative approaches, boosted by large-scale pre-trained video generation models, show promise in handling intricate scenes. We propose Motion-aware Generative frame (MoG) that synergizes intermediate flow guidance with generative capacities to enhance fidelity.
  arXiv  Detail & Related papers  (2025-01-07T11:03:43Z)
• KinMo: Kinematic-aware Human Motion Understanding and Generation [6.962697597686156]
  Controlling human motion based on text presents an important challenge in computer vision. Traditional approaches often rely on holistic action descriptions for motion synthesis. We propose a novel motion representation that decomposes motion into distinct body joint group movements.
  arXiv  Detail & Related papers  (2024-11-23T06:50:11Z)
• Real-time Diverse Motion In-betweening with Space-time Control [4.910937238451485]
  In this work, we present a data-driven framework for generating diverse in-betweening motions for kinematic characters. We demonstrate that our in-betweening approach can synthesize both locomotion and unstructured motions, enabling rich, versatile, and high-quality animation generation.
  arXiv  Detail & Related papers  (2024-09-30T22:45:53Z)
• Lagrangian Motion Fields for Long-term Motion Generation [51.02126882968116]
  We introduce the concept of Lagrangian Motion Fields, specifically designed for long-term motion generation.<n>By treating each joint as a Lagrangian particle with uniform velocity over short intervals, our approach condenses motion representations into a series of "supermotions"<n>Our solution is versatile and lightweight, eliminating the need for neural network preprocessing.
  arXiv  Detail & Related papers  (2024-09-03T01:38:06Z)
• Local Action-Guided Motion Diffusion Model for Text-to-Motion Generation [52.87672306545577]
  Existing motion generation methods primarily focus on the direct synthesis of global motions. We propose the local action-guided motion diffusion model, which facilitates global motion generation by utilizing local actions as fine-grained control signals. Our method provides flexibility in seamlessly combining various local actions and continuous guiding weight adjustment.
  arXiv  Detail & Related papers  (2024-07-15T08:35:00Z)
• MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model [29.93359157128045]
  This work introduces MotionLCM, extending controllable motion generation to a real-time level. We first propose the motion latent consistency model (MotionLCM) for motion generation, building upon the latent diffusion model. By adopting one-step (or few-step) inference, we further improve the runtime efficiency of the motion latent diffusion model for motion generation.
  arXiv  Detail & Related papers  (2024-04-30T17:59:47Z)
• FLD: Fourier Latent Dynamics for Structured Motion Representation and Learning [19.491968038335944]
  We introduce a self-supervised, structured representation and generation method that extracts spatial-temporal relationships in periodic or quasi-periodic motions. Our work opens new possibilities for future advancements in general motion representation and learning algorithms.
  arXiv  Detail & Related papers  (2024-02-21T13:59:21Z)
• Motion Flow Matching for Human Motion Synthesis and Editing [75.13665467944314]
  We propose emphMotion Flow Matching, a novel generative model for human motion generation featuring efficient sampling and effectiveness in motion editing applications. Our method reduces the sampling complexity from thousand steps in previous diffusion models to just ten steps, while achieving comparable performance in text-to-motion and action-to-motion generation benchmarks.
  arXiv  Detail & Related papers  (2023-12-14T12:57:35Z)
• DiffusionPhase: Motion Diffusion in Frequency Domain [69.811762407278]
  We introduce a learning-based method for generating high-quality human motion sequences from text descriptions. Existing techniques struggle with motion diversity and smooth transitions in generating arbitrary-length motion sequences. We develop a network encoder that converts the motion space into a compact yet expressive parameterized phase space.
  arXiv  Detail & Related papers  (2023-12-07T04:39:22Z)
• MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis [73.52948992990191]
  MoFusion is a new denoising-diffusion-based framework for high-quality conditional human motion synthesis. We present ways to introduce well-known kinematic losses for motion plausibility within the motion diffusion framework. We demonstrate the effectiveness of MoFusion compared to the state of the art on established benchmarks in the literature.
  arXiv  Detail & Related papers  (2022-12-08T18:59:48Z)
• Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
  We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs. Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
  arXiv  Detail & Related papers  (2022-12-08T03:07:00Z)
• MoDi: Unconditional Motion Synthesis from Diverse Data [51.676055380546494]
  We present MoDi, an unconditional generative model that synthesizes diverse motions. Our model is trained in a completely unsupervised setting from a diverse, unstructured and unlabeled motion dataset. We show that despite the lack of any structure in the dataset, the latent space can be semantically clustered.
  arXiv  Detail & Related papers  (2022-06-16T09:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.