MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation

• URL: http://arxiv.org/abs/2607.29180v1
• Date: Fri, 31 Jul 2026 09:01:25 GMT
• Title: MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation
• Abstract summary: Text-to-motion generation must produce motions that are semantically correct, temporally coherent, and physically plausible.<n>A natural approach is to first project motion data into a structured semantic space and then train a generative model within that space.<n>MoRAE addresses the two bottlenecks separately. A compact bottleneck distills the structured JEPA representation while removing weak and redundant directions.<n> Motion-coupled training then aligns the retained latent geometry with the decoder, making characteristic flow errors less costly after decoding.
• Score: 28.960999832402845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-motion generation must produce motions that are semantically correct, temporally coherent, and physically plausible. A natural approach is to first project motion data into a structured semantic space and then train a generative model within that space. Such a paradigm has been highly successful in image generation through Representation Autoencoders (RAEs), where a frozen self-supervised encoder provides semantic features for diffusion or flow models to learn from. However, direct transfer of such a paradigm to motion space using Motion-JEPA as the frozen encoder fails dramatically. We diagnose this failure geometrically and identify two motion-specific bottlenecks: (1) the JEPA feature space is spectrally ill-conditioned, making the Gaussian-to-data transport unstable; and (2) even with a well-conditioned spectrum, flow residuals tend to align with decoder-sensitive directions, where small latent errors are amplified into large motion artifacts after decoding. Based on these insights, we propose MoRAE. MoRAE addresses the two bottlenecks separately. A compact bottleneck distills the structured JEPA representation while removing weak and redundant directions, bringing the latent spectrum into a transport-stable regime. Motion-coupled training then aligns the retained latent geometry with the decoder, making characteristic flow errors less costly after decoding. With this flow-friendly latent, a standard non-autoregressive Flow-Matching DiT achieves state-of-the-art performance.

Related papers

• Drifting Models for Surrogate Flow Modeling [0.4925906256430175]
  We introduce a conditional architecture that performs drifting in a learned VAE latent space and uses label-aware masking to align generated samples with their boundary conditions.<n>Our model matches iterative diffusion in accuracy and flow consistency while running two orders of magnitude faster.
  arXiv  Detail & Related papers  (2026-06-05T17:35:09Z)
• SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation [73.51436199324066]
  Normalizing flows (NFs) provide exact likelihoods and deterministic invertible sampling, but have lagged behind diffusion models for large-scale image generation.<n>We propose SRC-Flow, which introduces a Semantic Representation (SRC) to compact high-dimensional RAE features into a low-dimensional semantic space.<n>SRC-Flow achieves state-of-the-art generation quality among flow methods, with gFID scores of 1.65 and 2.07 under computation-free guidance.
  arXiv  Detail & Related papers  (2026-05-18T12:03:41Z)
• Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation [34.87535133080741]
  Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks.<n>Current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error propagation during the iterative denoising process.<n>This paper introduces Reconstruction-Anchored Diffusion Model (RAM) to address these challenges.
  arXiv  Detail & Related papers  (2026-01-21T09:11:45Z)
• LUMA: Low-Dimension Unified Motion Alignment with Dual-Path Anchoring for Text-to-Motion Diffusion Model [18.564067196226436]
  We propose a text-to-motion diffusion model that incorporates dual-path anchoring to enhance semantic alignment.<n>LUMA achieves state-of-the-art performance, with FID scores of 0.035 and 0.123, respectively.
  arXiv  Detail & Related papers  (2025-09-29T17:58:28Z)
• Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression [97.66080040613726]
  We propose a Bidirectional Feature-aligned Motion Transformation (Bi-FMT) framework that implicitly models motion in the feature space.<n>Bi-FMT aligns features across both past and future frames to produce temporally consistent latent representations.<n>We show Bi-FMT surpasses D-DPCC and AdaDPCC in both compression efficiency and runtime.
  arXiv  Detail & Related papers  (2025-09-18T03:51:06Z)
• FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems [51.15230303652732]
  FLEX (F Low EXpert) is a backbone architecture for generative modeling of-temporal physical systems.<n>It reduces the variance of the velocity field in the diffusion model, which helps stabilize training.<n>It achieves accurate predictions for super-resolution and forecasting tasks using as few features as two reverse diffusion steps.
  arXiv  Detail & Related papers  (2025-05-23T00:07:59Z)
• Complexity Matters: Rethinking the Latent Space for Generative Modeling [65.64763873078114]
  In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion. In this study, we aim to shed light on this under-explored topic by rethinking the latent space from the perspective of model complexity.
  arXiv  Detail & Related papers  (2023-07-17T07:12:29Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.