MotionPyramid: Hierarchical Motion Representation and Residual Interfaces

• URL: http://arxiv.org/abs/2606.20705v1
• Date: Mon, 15 Jun 2026 18:00:11 GMT
• Title: MotionPyramid: Hierarchical Motion Representation and Residual Interfaces
• Abstract summary: In humanoid control, low level actions specify immediate motor commands, while meaningful behavior is organized over longer temporal scales.<n>We introduce MotionPyramid, a hierarchical action representation that learns such structure from motion data.<n>We show MotionPyramid can be organized into a reusable multi level representation, providing structured abstraction without sacrificing controllability.
• Score: 9.880977839912811
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We ask whether the representational hierarchy seen in perception, from local primitives such as edges to higher level structures such as parts and objects, can be established for motion. In humanoid control, low level actions specify immediate motor commands, while meaningful behavior is organized over longer temporal scales, including contacts, gait fragments, balance recovery, reaching, and whole body skills. We introduce MotionPyramid, a hierarchical action representation that learns such structure from motion data. Starting from a motion tracking teacher, it trains a recursive stack of latent decoders: low level latents decode to immediate full body motor commands, while higher level latents unfold through lower levels into temporally extended motion programs. After pretraining, the hierarchy is frozen and reused by downstream reinforcement learning policies as a family of action interfaces at different control resolutions. Experiments show the learned levels form a motion hierarchy: coarser interfaces improve early learning and motion regularity by constraining exploration to structured segments, while finer interfaces preserve feedback control and final task precision. Representation probes show the hierarchy supports traversal, interpolation, transition, and qualitative composition, exposing editable control handles across temporal scales. Finally, we introduce Residual Interfaces, letting a downstream policy maintain coarse, segment level, and frame level residual commands through the frozen hierarchy. Analogous to residual or skip connections in deep networks, this allows coarse motion programs and fine residual corrections to coexist within one controller. MotionPyramid shows that motion, like perception, can be organized into a reusable multi level representation, providing structured abstraction without sacrificing controllability.

Related papers

• Prior-First, Condition-Second: Scalable and Controllable Hand Motion Completion [27.154998447721102]
  We propose a prior-first, condition-second framework for body-conditioned hand motion completion.<n>Our framework generates coherent, kinematically consistent hand motion from body dynamics in real time.<n>Our framework improves kinematic plausibility, robustness, and controllability compared to end-to-end conditioned baselines.
  arXiv  Detail & Related papers  (2026-07-07T07:40:13Z)
• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories [62.54876287800644]
  Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric.<n>Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction.<n>We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable.
  arXiv  Detail & Related papers  (2026-06-21T14:25:50Z)
• MultiAct: Text-to-Motion Generation from Composite Text via Tailored Attention Guidance [54.131431576597514]
  MultiAct is an unpaired, inference-time framework for compositional text-to-motion synthesis.<n>Our method counteracts semantic collapse by adaptively amplifying cross-attention scores associated with underrepresented prompt components.
  arXiv  Detail & Related papers  (2026-05-29T07:14:48Z)
• Online Continual Learning with Dynamic Label Hierarchies [72.33335166136045]
  We introduce a new problem setting, DHOCL, where severity evolve across granularities and each sample provides supervision at a single hierarchical level.<n>In this setting, we find two fundamental issues: (i) partial supervision under mixed granularities provides only point-wise signals over an evolving path-wise hierarchy, which constrains plasticity and undermines cross-level semantic consistency.<n>We propose HALO, which adaptively combines complementary classification heads, regularized by organized learnable hierarchical prototypes, enabling rapid adaptation, hierarchical consistency, and structured knowledge consolidation as the taxonomy evolves.
  arXiv  Detail & Related papers  (2026-05-12T08:20:23Z)
• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• MotionHiFlow: Text-to-motion via hierarchical flow matching [23.3063383764363]
  Text-to-motion generation aims to generate 3D human motions that are tightly aligned with the input text.<n>We propose textitMotionHiFlow, a hierarchical flow matching framework to generate motion progressively.<n>The flows at lower scales capture high-level semantics and coarse motion structures, while flows at higher scales refine temporal details.
  arXiv  Detail & Related papers  (2026-04-25T12:16:37Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• KinMo: Kinematic-aware Human Motion Understanding and Generation [6.962697597686156]
  Current human motion synthesis frameworks rely on global action descriptions.<n>A single coarse description, such as run, fails to capture details such as variations in speed, limb positioning, and kinematic dynamics.<n>We introduce KinMo, a unified framework built on a hierarchical describable motion representation.
  arXiv  Detail & Related papers  (2024-11-23T06:50:11Z)
• Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs [31.244039305932287]
  We propose hierarchical semantic graphs for fine-grained control over motion generation. We disentangle motion descriptions into hierarchical semantic graphs including three levels of motions, actions, and specifics. Our method can continuously refine the generated motion, which may have a far-reaching impact on the community.
  arXiv  Detail & Related papers  (2023-11-02T06:20:23Z)
• Motion In-Betweening with Phase Manifolds [29.673541655825332]
  This paper introduces a novel data-driven motion in-betweening system to reach target poses of characters by making use of phases variables learned by a Periodic Autoencoder. Our approach utilizes a mixture-of-experts neural network model, in which the phases cluster movements in both space and time with different expert weights.
  arXiv  Detail & Related papers  (2023-08-24T12:56:39Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)
• Hierarchical Contrastive Motion Learning for Video Action Recognition [100.9807616796383]
  We present hierarchical contrastive motion learning, a new self-supervised learning framework to extract effective motion representations from raw video frames. Our approach progressively learns a hierarchy of motion features that correspond to different abstraction levels in a network. Our motion learning module is lightweight and flexible to be embedded into various backbone networks.
  arXiv  Detail & Related papers  (2020-07-20T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.