Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors

• URL: http://arxiv.org/abs/2603.29655v1
• Date: Tue, 31 Mar 2026 12:18:57 GMT
• Title: Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors
• Abstract summary: Masked generative models have become a strong paradigm for text-to-motion synthesis, but they still treat motion frames too uniformly.<n>We show that current masked motion generators degrade disproportionately on dynamically complex motions.<n>Motivated by this mismatch, we introduce the Motion Spectral Descriptor (MSD), a simple and parameter-free measure of local dynamic complexity.
• Score: 10.685712212496753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Masked generative models have become a strong paradigm for text-to-motion synthesis, but they still treat motion frames too uniformly during masking, attention, and decoding. This is a poor match for motion, where local dynamic complexity varies sharply over time. We show that current masked motion generators degrade disproportionately on dynamically complex motions, and that frame-wise generation error is strongly correlated with motion dynamics. Motivated by this mismatch, we introduce the Motion Spectral Descriptor (MSD), a simple and parameter-free measure of local dynamic complexity computed from the short-time spectrum of motion velocity. Unlike learned difficulty predictors, MSD is deterministic, interpretable, and derived directly from the motion signal itself. We use MSD to make masked motion generation complexity-aware. In particular, MSD guides content-focused masking during training, provides a spectral similarity prior for self-attention, and can additionally modulate token-level sampling during iterative decoding. Built on top of masked motion generators, our method, DynMask, improves motion generation most clearly on dynamically complex motions while also yielding stronger overall FID on HumanML3D and KIT-ML. These results suggest that respecting local motion complexity is a useful design principle for masked motion generation. Project page: https://xiangyue-zhang.github.io/DynMask

Related papers

• Following Motion for Sequential Modeling in Video Frame Interpolation [22.71397762551783]
  State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI)<n>We propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI.
  arXiv  Detail & Related papers  (2026-08-24T06:43:48Z)
• MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation [18.165045289069703]
  Motion-Aligned Selective Scan (MASS) is a novel framework that reformulates feature scanning from static spatial grids to dynamic motion trajectories.<n>MASS builds a feature sequence along each pixel's flow-guided trajectory and aggregates it with an SSM.
  arXiv  Detail & Related papers  (2026-06-26T04:56:31Z)
• Follow Your Track: Precise Skeleton Animation Controlled by 3D Trajectories [93.39170866007431]
  4D generation aims to animate 3D objects with realistic motion, holding great promise for applications.<n>Existing methods typically decouple 3D asset generation from motion synthesis.<n>We present textttACT, a trajectory-conditioned framework for topology-general skeletal animation.
  arXiv  Detail & Related papers  (2026-06-24T03:18:10Z)
• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration [66.81078857385698]
  CoMoGen is a controllable video generation framework that generates realistic interactive dynamics from a single binary mask sequence conditioned on an input image.<n>CoMoGen consistently outperforms prior controllable video generation methods and achieves state-of-the-art performance in motion fidelity and perceptual realism.
  arXiv  Detail & Related papers  (2026-05-21T19:51:20Z)
• IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation [58.297199313494]
  Implicit methods capture motion semantics directly from driving video, but suffer from identity leakage and entanglement between motion and appearance.<n>We propose a novel implicit motion representation that compresses per-frame motion into compact 1D motion tokens.<n>Our methodology employs a three-stage training strategy to enhance the training efficiency and ensure high fidelity.
  arXiv  Detail & Related papers  (2026-02-07T11:17:20Z)
• M2DAO-Talker: Harmonizing Multi-granular Motion Decoupling and Alternating Optimization for Talking-head Generation [65.48046909056468]
  We reformulate talking head generation into a unified framework comprising video preprocessing, motion representation, and rendering reconstruction.<n>M2DAO-Talker achieves state-of-the-art performance, with the 2.43 dB PSNR improvement in generation quality and 0.64 gain in user-evaluated video realness.
  arXiv  Detail & Related papers  (2025-07-11T04:48:12Z)
• Motion Anything: Any to Motion Generation [24.769413146731264]
  Motion Anything is a multimodal motion generation framework.<n>Our model adaptively encodes multimodal conditions, including text and music, improving controllability.<n>Text-Music-Dance dataset consists of 2,153 pairs of text, music, and dance, making it twice the size of AIST++.
  arXiv  Detail & Related papers  (2025-03-10T06:04:31Z)
• BAMM: Bidirectional Autoregressive Motion Model [14.668729995275807]
  Bidirectional Autoregressive Motion Model (BAMM) is a novel text-to-motion generation framework. BAMM consists of two key components: a motion tokenizer that transforms 3D human motion into discrete tokens in latent space, and a masked self-attention transformer that autoregressively predicts randomly masked tokens. This feature enables BAMM to simultaneously achieving high-quality motion generation with enhanced usability and built-in motion editability.
  arXiv  Detail & Related papers  (2024-03-28T14:04:17Z)
• Spectral Motion Alignment for Video Motion Transfer using Diffusion Models [54.32923808964701]
  Spectral Motion Alignment (SMA) is a framework that refines and aligns motion vectors using Fourier and wavelet transforms.<n> SMA learns motion patterns by incorporating frequency-domain regularization, facilitating the learning of whole-frame global motion dynamics.<n>Extensive experiments demonstrate SMA's efficacy in improving motion transfer while maintaining computational efficiency and compatibility across various video customization frameworks.
  arXiv  Detail & Related papers  (2024-03-22T14:47:18Z)
• MoStGAN-V: Video Generation with Temporal Motion Styles [28.082294960744726]
  Previous works attempt to generate videos in arbitrary lengths either in an autoregressive manner or regarding time as a continuous signal. We argue that a single time-agnostic latent vector of style-based generator is insufficient to model various and temporally-consistent motions. We introduce additional time-dependent motion styles to model diverse motion patterns.
  arXiv  Detail & Related papers  (2023-04-05T22:47:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.