Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editing

• URL: http://arxiv.org/abs/2512.22464v1
• Date: Sat, 27 Dec 2025 04:45:31 GMT
• Title: Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editing
• Abstract summary: We introduce pose-guided residual refinement for motion (PGR$2$M), a hybrid representation that augments interpretable pose codes with residual codes learned via residual vector quantization (RVQ)<n>A base Transformer autoregressively predicts pose codes from text, and a refine Transformer predicts residual codes conditioned on text, pose codes, and quantization stage.<n>Experiments on HumanML3D and KIT-ML show that PGR$2$M improves Fréchet distance and reconstruction metrics for both generation and editing compared with CoMo.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-based 3D motion generation aims to automatically synthesize diverse motions from natural-language descriptions to extend user creativity, whereas motion editing modifies an existing motion sequence in response to text while preserving its overall structure. Pose-code-based frameworks such as CoMo map quantifiable pose attributes into discrete pose codes that support interpretable motion control, but their frame-wise representation struggles to capture subtle temporal dynamics and high-frequency details, often degrading reconstruction fidelity and local controllability. To address this limitation, we introduce pose-guided residual refinement for motion (PGR$^2$M), a hybrid representation that augments interpretable pose codes with residual codes learned via residual vector quantization (RVQ). A pose-guided RVQ tokenizer decomposes motion into pose latents that encode coarse global structure and residual latents that model fine-grained temporal variations. Residual dropout further discourages over-reliance on residuals, preserving the semantic alignment and editability of the pose codes. On top of this tokenizer, a base Transformer autoregressively predicts pose codes from text, and a refine Transformer predicts residual codes conditioned on text, pose codes, and quantization stage. Experiments on HumanML3D and KIT-ML show that PGR$^2$M improves Fréchet inception distance and reconstruction metrics for both generation and editing compared with CoMo and recent diffusion- and tokenization-based baselines, while user studies confirm that it enables intuitive, structure-preserving motion edits.

Related papers

• Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editing [78.44034844227868]
  Text-driven human motion editing aims to modify existing motion sequences according to natural language instructions.<n>Change and Invariance Motion Editing (CIME) is a unified framework that comprehensively decouples change and invariance into spatial pose and temporal rhythm dimensions.<n>Experiments on the MotionFix and STANCE Adjustment datasets demonstrate that CIME achieves state-of-the-art performance in editing alignment and structural fidelity.
  arXiv  Detail & Related papers  (2026-08-17T01:54:44Z)
• In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics [109.99871326497333]
  In-Context Model Predictive Generation (ICMPG) is a framework that integrates language-model planning with inference-time physical feedback.<n>ICMPG reformulates motion synthesis as a Model Predictive Control (MPC)-like process with two modules.
  arXiv  Detail & Related papers  (2026-06-25T12:50:33Z)
• A$^{2}$V-SLP: Alignment-Aware Variational Modeling for Disentangled Sign Language Production [0.9384603486206738]
  A$2$V-SLP learns articulator-wise disentangled latent distributions rather than deterministic embeddings.<n>A disentangled Variational Autoencoder encodes ground-truth sign pose sequences and extracts articulator-specific mean and variance vectors.
  arXiv  Detail & Related papers  (2026-02-12T12:07:32Z)
• Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation [34.87535133080741]
  Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks.<n>Current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error propagation during the iterative denoising process.<n>This paper introduces Reconstruction-Anchored Diffusion Model (RAM) to address these challenges.
  arXiv  Detail & Related papers  (2026-01-21T09:11:45Z)
• Error-Propagation-Free Learned Video Compression With Dual-Domain Progressive Temporal Alignment [92.57576987521107]
  We propose a novel unifiedtransform framework with dual-domain progressive temporal alignment and quality-conditioned mixture-of-expert (QCMoE)<n>QCMoE allows continuous and consistent rate control with appealing R-D performance.<n> Experimental results show that the proposed method achieves competitive R-D performance compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2025-12-11T09:14:51Z)
• VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction [83.50898344094153]
  VQRAE produces Continuous semantic features for image understanding and tokens for visual generation within a unified tokenizer.<n>Design enables negligible semantic information for maintaining the ability of multimodal understanding, discrete tokens.<n>VQRAE presents competitive performance on several benchmarks of visual understanding, generation and reconstruction.
  arXiv  Detail & Related papers  (2025-11-28T17:26:34Z)
• Making Pose Representations More Expressive and Disentangled via Residual Vector Quantization [0.0]
  We propose a method that augments pose code-based latent representations with continuous motion features using residual vector quantization (RVQ)<n>This design preserves the interpretability and manipulability of pose codes while effectively capturing subtle motion characteristics such as high-frequency details.
  arXiv  Detail & Related papers  (2025-08-20T09:29:21Z)
• MOGO: Residual Quantized Hierarchical Causal Transformer for High-Quality and Real-Time 3D Human Motion Generation [3.6669020073583756]
  MOGO is a novel autoregressive framework tailored for efficient and real-time 3D motion generation.<n>MoGO comprises two key components: MoSA-VQ, a motion scale-adaptive residual vector quantization module, and RQHC-Transformer, a residual quantized hierarchical causal transformer.<n>To enhance semantic fidelity, we introduce a text condition alignment mechanism that improves motion decoding under textual control.
  arXiv  Detail & Related papers  (2025-06-06T10:26:54Z)
• Absolute Coordinates Make Motion Generation Easy [8.153961351540834]
  State-of-the-art text-to-motion generation models rely on the kinematic-aware, local-relative motion representation popularized by HumanML3D.<n>We propose a radically simplified and long-abandoned alternative for text-to-motion generation: absolute joint coordinates in global space.
  arXiv  Detail & Related papers  (2025-05-26T00:36:00Z)
• CoMo: Controllable Motion Generation through Language Guided Pose Code Editing [57.882299081820626]
  We introduce CoMo, a Controllable Motion generation model, adept at accurately generating and editing motions. CoMo decomposes motions into discrete and semantically meaningful pose codes. It autoregressively generates sequences of pose codes, which are then decoded into 3D motions.
  arXiv  Detail & Related papers  (2024-03-20T18:11:10Z)
• SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes [59.23385953161328]
  Novel view synthesis for dynamic scenes is still a challenging problem in computer vision and graphics. We propose a new representation that explicitly decomposes the motion and appearance of dynamic scenes into sparse control points and dense Gaussians. Our method can enable user-controlled motion editing while retaining high-fidelity appearances.
  arXiv  Detail & Related papers  (2023-12-04T11:57:14Z)
• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.