MotionMaestro: Masked Tokenization for Unified Motion Generation
Abstract Overview
MotionMaestro is a unified human motion generation framework that treats diverse motion-generation tasks as different observation patterns over a motion sequence. It learns a shared latent representation for full motions and heterogeneous partial observations through task-aligned masked motion tokenization, followed by a conditional flow-matching generator. The model uses an explicit observation map and an observation loss to indicate which motion features are given and to encourage preservation of supplied conditions during generation. In one model, it supports unconditional and text-guided synthesis, pose conditioning, interpolation, extrapolation, partial completion, trajectory control, and long-sequence generation, and it is evaluated on the large-scale RoMo and MotionMillion datasets.
Novelty
The paper's main novelty is formulating diverse motion-generation tasks as structured masking patterns over motion sequences and training the tokenizer accordingly, allowing full targets and partial conditions to share the same latent space. It also introduces an observation map and an observation-space loss to explicitly identify observed motion features and enforce condition preservation during generation.
Results
On RoMo, MotionMaestro achieves the best results across all ten evaluated tasks, and on MotionMillion it achieves the best results on eight of ten tasks. It notably improves frame interpolation quality and error relative to unified baselines, and ablation studies show that structured masking, the observation map, and the observation loss each contribute to performance and condition fidelity.
Key Points
- MotionMaestro unifies a broad set of motion-generation tasks by representing them as different temporal, spatial, and feature-group observation masks over the same motion sequence.
- The method combines a three-stage masked tokenizer training procedure with a conditional flow-matching generator, plus an observation map and observation loss to preserve provided motion constraints.
- Experiments on RoMo and MotionMillion show strong quantitative performance versus MDM, OmniControl, and MotionLab, especially for interpolation, long-sequence generation, and several condition-controlled tasks.