Making Pose Representations More Expressive and Disentangled via Residual Vector Quantization

• URL: http://arxiv.org/abs/2508.14561v1
• Date: Wed, 20 Aug 2025 09:29:21 GMT
• Title: Making Pose Representations More Expressive and Disentangled via Residual Vector Quantization
• Abstract summary: We propose a method that augments pose code-based latent representations with continuous motion features using residual vector quantization (RVQ)<n>This design preserves the interpretability and manipulability of pose codes while effectively capturing subtle motion characteristics such as high-frequency details.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent progress in text-to-motion has advanced both 3D human motion generation and text-based motion control. Controllable motion generation (CoMo), which enables intuitive control, typically relies on pose code representations, but discrete pose codes alone cannot capture fine-grained motion details, limiting expressiveness. To overcome this, we propose a method that augments pose code-based latent representations with continuous motion features using residual vector quantization (RVQ). This design preserves the interpretability and manipulability of pose codes while effectively capturing subtle motion characteristics such as high-frequency details. Experiments on the HumanML3D dataset show that our model reduces Frechet inception distance (FID) from 0.041 to 0.015 and improves Top-1 R-Precision from 0.508 to 0.510. Qualitative analysis of pairwise direction similarity between pose codes further confirms the model's controllability for motion editing.

Related papers

• MotionVLA: Vision-Language-Action Model for Humanoid Motion [54.785960777274276]
  We present DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams.<n>We also present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence.
  arXiv  Detail & Related papers  (2026-06-13T06:10:48Z)
• DrawMotion: Generating 3D Human Motions by Freehand Drawing [64.44582376360027]
  This paper introduces DrawMotion, an efficient diffusion-based framework designed for multi-condition scenarios.<n>We tackle the fine-grained motion generation task from three perspectives: 1) freehand drawing condition, 2) multi-condition fusion, and 3) training-free guidance.<n>Experiments and user studies demonstrate that the freehand drawing approach reduces user time by approximately 46.7% when generating motions aligned with their imagination.
  arXiv  Detail & Related papers  (2026-05-20T09:43:50Z)
• FG-Portrait: 3D Flow Guided Editable Portrait Animation [52.51549429089156]
  Motion transfer from the driving to the source portrait remains a key challenge in portrait animation.<n>Current diffusion-based approaches condition only on the driving motion, which fails to capture source-to-driving correspondences.<n>We introduce 3D flows, a learning-free and geometry-driven motion correspondence directly computed from parametric 3D head models.
  arXiv  Detail & Related papers  (2026-03-24T16:09:19Z)
• GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry [61.24189040578178]
  We propose a fully learning-based approach that directly infers moving objects from latent feature representations via attention mechanisms.<n>Our key insight is to bypass explicit correspondence estimation and instead let the model learn to implicitly disentangle object and camera motion.<n>Our approach achieves state-of-the-art motion segmentation performance with high efficiency.
  arXiv  Detail & Related papers  (2026-02-25T11:36:33Z)
• IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation [58.297199313494]
  Implicit methods capture motion semantics directly from driving video, but suffer from identity leakage and entanglement between motion and appearance.<n>We propose a novel implicit motion representation that compresses per-frame motion into compact 1D motion tokens.<n>Our methodology employs a three-stage training strategy to enhance the training efficiency and ensure high fidelity.
  arXiv  Detail & Related papers  (2026-02-07T11:17:20Z)
• Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editing [0.0]
  We introduce pose-guided residual refinement for motion (PGR$2$M), a hybrid representation that augments interpretable pose codes with residual codes learned via residual vector quantization (RVQ)<n>A base Transformer autoregressively predicts pose codes from text, and a refine Transformer predicts residual codes conditioned on text, pose codes, and quantization stage.<n>Experiments on HumanML3D and KIT-ML show that PGR$2$M improves Fréchet distance and reconstruction metrics for both generation and editing compared with CoMo.
  arXiv  Detail & Related papers  (2025-12-27T04:45:31Z)
• From Tokens to Nodes: Semantic-Guided Motion Control for Dynamic 3D Gaussian Splatting [26.57713792657793]
  We propose a motion-adaptive framework that aligns control density with motion complexity.<n>We show significant improvements in reconstruction quality and efficiency over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-03T05:33:58Z)
• Towards Robust and Controllable Text-to-Motion via Masked Autoregressive Diffusion [33.9786226622757]
  We propose a robust motion generation framework MoMADiff to generate 3D human motion from text descriptions.<n>Our model supports flexible user-provided specification, enabling precise control over both spatial and temporal aspects of motion synthesis.<n>Our method consistently outperforms state-of-the-art models in motion quality, instruction fidelity, and adherence.
  arXiv  Detail & Related papers  (2025-05-16T09:06:15Z)
• Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors [25.67875816218477]
  Full-body pose estimation from sparse tracking signals is not limited by environmental conditions or recording range.<n>Previous works either face the challenge of wearing additional sensors on the pelvis and lower-body or rely on external visual sensors to obtain global positions of key joints.<n>To improve the practicality of the technology for virtual reality applications, we estimate full-body poses using only inertial data obtained from three Inertial Measurement Unit (IMU) sensors worn on the head and wrists.
  arXiv  Detail & Related papers  (2025-05-08T15:28:09Z)
• Articulate That Object Part (ATOP): 3D Part Articulation via Text and Motion Personalization [12.944411575346528]
  ATOP (Articulate That Object Part) is a novel few-shot method based on motion personalization to articulate a static 3D object with respect to a part and its motion as prescribed in a text prompt.<n>We show that our method can generate realistic motion samples with higher accuracy, leading to more generalizable 3D motion predictions.
  arXiv  Detail & Related papers  (2025-02-11T05:47:16Z)
• Exploring Vision Transformers for 3D Human Motion-Language Models with Motion Patches [12.221087476416056]
  We introduce "motion patches", a new representation of motion sequences, and propose using Vision Transformers (ViT) as motion encoders via transfer learning. These motion patches, created by dividing and sorting skeleton joints based on motion sequences, are robust to varying skeleton structures. We find that transfer learning with pre-trained weights of ViT obtained through training with 2D image data can boost the performance of motion analysis.
  arXiv  Detail & Related papers  (2024-05-08T02:42:27Z)
• SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer [57.506654943449796]
  We propose an efficient, sparse-controlled video-to-4D framework named SC4D that decouples motion and appearance. Our method surpasses existing methods in both quality and efficiency. We devise a novel application that seamlessly transfers motion onto a diverse array of 4D entities.
  arXiv  Detail & Related papers  (2024-04-04T18:05:18Z)
• Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
  We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs. Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
  arXiv  Detail & Related papers  (2022-12-08T03:07:00Z)
• MotionBERT: A Unified Perspective on Learning Human Motion Representations [46.67364057245364]
  We present a unified perspective on tackling various human-centric video tasks by learning human motion representations from large-scale and heterogeneous data resources. We propose a pretraining stage in which a motion encoder is trained to recover the underlying 3D motion from noisy partial 2D observations. We implement motion encoder with a Dual-stream Spatio-temporal Transformer (DSTformer) neural network.
  arXiv  Detail & Related papers  (2022-10-12T19:46:25Z)
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
  We introduce a unified deep generative network for both diverse and controllable motion prediction. Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
  arXiv  Detail & Related papers  (2021-08-19T00:58:00Z)
• HuMoR: 3D Human Motion Model for Robust Pose Estimation [100.55369985297797]
  HuMoR is a 3D Human Motion Model for Robust Estimation of temporal pose and shape. We introduce a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence. We demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset.
  arXiv  Detail & Related papers  (2021-05-10T21:04:55Z)
• Residual Frames with Efficient Pseudo-3D CNN for Human Action Recognition [10.185425416255294]
  We propose to use residual frames as an alternative "lightweight" motion representation. We also develop a new pseudo-3D convolution module which decouples 3D convolution into 2D and 1D convolution.
  arXiv  Detail & Related papers  (2020-08-03T17:40:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.