LS-GAN: Human Motion Synthesis with Latent-space GANs

• URL: http://arxiv.org/abs/2501.01449v1
• Date: Mon, 30 Dec 2024 05:44:38 GMT
• Title: LS-GAN: Human Motion Synthesis with Latent-space GANs
• Abstract summary: Conditioned Motion synthesis takes a text input and outputs a 3D motion corresponding to the text.<n>We introduce a novel framework that utilizes Generative Adversarial Networks (GANs) in the latent space to enable faster training and inference.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Human motion synthesis conditioned on textual input has gained significant attention in recent years due to its potential applications in various domains such as gaming, film production, and virtual reality. Conditioned Motion synthesis takes a text input and outputs a 3D motion corresponding to the text. While previous works have explored motion synthesis using raw motion data and latent space representations with diffusion models, these approaches often suffer from high training and inference times. In this paper, we introduce a novel framework that utilizes Generative Adversarial Networks (GANs) in the latent space to enable faster training and inference while achieving results comparable to those of the state-of-the-art diffusion methods. We perform experiments on the HumanML3D, HumanAct12 benchmarks and demonstrate that a remarkably simple GAN in the latent space achieves a FID of 0.482 with more than 91% in FLOPs reduction compared to latent diffusion model. Our work opens up new possibilities for efficient and high-quality motion synthesis using latent space GANs.

Related papers

• ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment [38.82543734940858]
  Text-to-motion generation holds immense potential for applications in gaming, film, and robotics.<n>There exists a misalignment between text and motion distributions in diffusion models, which leads to semantically inconsistent motions.<n>We propose Reward-guided sampling Alignment (ReAlign) to address this limitation.<n>Our approach significantly improves text-motion alignment and motion quality compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-11-24T15:23:36Z)
• Geometric Neural Distance Fields for Learning Human Motion Priors [51.99890740169883]
  We introduce a novel 3D generative human motion prior that enables robust, temporally consistent, and physically plausible 3D motion recovery.<n>Our experiments show significant and consistent gains: trained on the AMASS dataset, NRMF remarkably generalizes across multiple input modalities.
  arXiv  Detail & Related papers  (2025-09-11T17:58:18Z)
• Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis [80.50342609047091]
  Scene-aware text-to-human synthesis generates diverse indoor motion samples from the same textual description. We propose a cascaded generation strategy that factorizes text-driven scene-specific human motion generation into three stages. We jointly improve realistic human motion synthesis and robust human motion analysis in 3D scenes.
  arXiv  Detail & Related papers  (2025-03-01T06:56:58Z)
• MotionPCM: Real-Time Motion Synthesis with Phased Consistency Model [7.920981293972071]
  Diffusion models have become a popular choice for human motion synthesis due to their powerful generative capabilities. Their high computational complexity and large sampling steps pose challenges for real-time applications. We introduce textbfMotionPCM, a phased consistency model-based approach designed to improve the quality and efficiency for real-time synthesis motion in latent space.
  arXiv  Detail & Related papers  (2025-01-31T12:17:04Z)
• LEAD: Latent Realignment for Human Motion Diffusion [12.40712030002265]
  Our goal is to generate realistic human motion from natural language. For motion synthesis, we evaluate LEAD on HumanML3D and KIT-ML and show comparable performance to the state-of-the-art in terms of realism, diversity, and text-motion consistency. For motion textual inversion, our method demonstrates improved capacity in capturing out-of-distribution characteristics in comparison to traditional VAEs.
  arXiv  Detail & Related papers  (2024-10-18T14:43:05Z)
• MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds [20.83684434910106]
  We present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Specifically, we propose novel decoupled joint acceleration to model human dynamics from existing limited motion data. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks.
  arXiv  Detail & Related papers  (2024-09-01T15:00:16Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• AMD:Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion [11.689663297469945]
  We propose the Adaptable Motion Diffusion model. It exploits a Large Language Model (LLM) to parse the input text into a sequence of concise and interpretable anatomical scripts. We then devise a two-branch fusion scheme that balances the influence of the input text and the anatomical scripts on the inverse diffusion process.
  arXiv  Detail & Related papers  (2023-12-20T04:49:45Z)
• Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model [60.27825196999742]
  We propose a novel Basic-to-Advanced Hierarchical Diffusion Model, named B2A-HDM, to collaboratively exploit low-dimensional and high-dimensional diffusion models for detailed motion synthesis. Specifically, the basic diffusion model in low-dimensional latent space provides the intermediate denoising result that is consistent with the textual description. The advanced diffusion model in high-dimensional latent space focuses on the following detail-enhancing denoising process.
  arXiv  Detail & Related papers  (2023-12-18T06:30:39Z)
• DiffusionPhase: Motion Diffusion in Frequency Domain [69.811762407278]
  We introduce a learning-based method for generating high-quality human motion sequences from text descriptions. Existing techniques struggle with motion diversity and smooth transitions in generating arbitrary-length motion sequences. We develop a network encoder that converts the motion space into a compact yet expressive parameterized phase space.
  arXiv  Detail & Related papers  (2023-12-07T04:39:22Z)
• MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis [73.52948992990191]
  MoFusion is a new denoising-diffusion-based framework for high-quality conditional human motion synthesis. We present ways to introduce well-known kinematic losses for motion plausibility within the motion diffusion framework. We demonstrate the effectiveness of MoFusion compared to the state of the art on established benchmarks in the literature.
  arXiv  Detail & Related papers  (2022-12-08T18:59:48Z)
• Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
  We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs. Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
  arXiv  Detail & Related papers  (2022-12-08T03:07:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.