Prior-First, Condition-Second: Scalable and Controllable Hand Motion Completion

• URL: http://arxiv.org/abs/2607.05938v1
• Date: Tue, 07 Jul 2026 07:40:13 GMT
• Title: Prior-First, Condition-Second: Scalable and Controllable Hand Motion Completion
• Abstract summary: We propose a prior-first, condition-second framework for body-conditioned hand motion completion.<n>Our framework generates coherent, kinematically consistent hand motion from body dynamics in real time.<n>Our framework improves kinematic plausibility, robustness, and controllability compared to end-to-end conditioned baselines.
• Score: 27.154998447721102
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthesizing hand motion that matches the full body motion and the semantic labels is a difficult task due to their high degrees of freedom and the lack of semantic labels. To cope with this issue, we propose a prior-first, condition-second framework for body-conditioned hand motion completion. Our framework first learns a generic body-hand kinematic prior from large-scale unstructured and unlabeled motion data, capturing the intrinsic coordination between global body dynamics and hand articulation. Semantic control is then introduced through lightweight adaptation on top of the frozen prior, avoiding the need to relearn kinematic structure for each control interface. Our framework centers on a streaming, autoregressive body-hand prior that generates coherent, kinematically consistent hand motion from body dynamics in real time, using structured kinematic modeling to maintain mechanical body-hand coupling. To enable practical controllability under limited supervision, we introduce semantically-layered adapters that inject conditioning signals at appropriate kinematic levels, supporting both self-supervised attribute control and weakly supervised text-driven control with only a few hours of labeled data. Extensive evaluations demonstrate that our framework improves kinematic plausibility, robustness, and controllability compared to end-to-end conditioned baselines, particularly in low-resource and cross-dataset settings. We further showcase real-time inference and an interactive authoring workflow, highlighting the applicability to production animation pipelines. Homepage: https://AIGAnimation.github.io/HandPrior/

Related papers

• MotionPyramid: Hierarchical Motion Representation and Residual Interfaces [9.880977839912811]
  In humanoid control, low level actions specify immediate motor commands, while meaningful behavior is organized over longer temporal scales.<n>We introduce MotionPyramid, a hierarchical action representation that learns such structure from motion data.<n>We show MotionPyramid can be organized into a reusable multi level representation, providing structured abstraction without sacrificing controllability.
  arXiv  Detail & Related papers  (2026-06-15T18:00:11Z)
• SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control [72.5651722107621]
  SCRIPT is a scalable diffusion policy with a multi-stage training framework for language-driven physics-based humanoid control.<n>The core of SCRIPT is a Joint Action-State-Text Diffusion Transformer (JAST-DiT), which represents actions, physical states, and text as dedicated token streams.<n>To stabilize autore control, we introduce a nonlinear history conditioning mechanism, which preserves the dense recent context and samples increasingly sparse cues from long-term history.
  arXiv  Detail & Related papers  (2026-05-21T14:17:21Z)
• MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation [66.66098171359995]
  MotionMERGE is a unified framework that bridges the granularity gap in motion-language models.<n>First, we pioneer the study of fine-grained language-guided motion control, including detailed understanding and localized editing.<n>Second, we design ReasoningAware Granularity-Synergy pre-training, a novel strategy that employs joint supervision for cross-granularity alignment.<n>Third, we curate MotionFineEdit, a large-scale dataset with the first finegrained-temporal corrective instructions and motion-grounded CoT annotations.
  arXiv  Detail & Related papers  (2026-05-18T18:00:04Z)
• From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation [43.4347713269043]
  Action-conditioned surgical video generation is a critical yet highly challenging problem for robotic surgery.<n>We propose a kinematic-to-visual lifting paradigm that converts articulated kinematics into a unified set of five image-aligned control modalities.<n>Our method consistently improves action faithfulness, visual fidelity, and cross-domain generalization over diverse baselines.
  arXiv  Detail & Related papers  (2026-05-09T05:48:51Z)
• HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching [113.81911881001905]
  HO-Flow is a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects.<n>It first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold.<n>It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation.
  arXiv  Detail & Related papers  (2026-04-12T22:06:11Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation [50.792027578906804]
  We introduce SteadyDancer, an Image-to-Video (R2V) paradigm-based framework that achieves harmonized and coherent animation.<n> Experiments demonstrate that SteadyDancer achieves state-of-the-art performance in both appearance fidelity and motion control.
  arXiv  Detail & Related papers  (2025-11-24T17:15:55Z)
• Motion Synthesis with Sparse and Flexible Keyjoint Control [10.592822014277631]
  We propose a controllable motions synthesis framework that respects sparse and flexible keyjoint signals.<n>We demonstrate the effectiveness of sparse and flexible keyjoint control through comprehensive experiments on diverse datasets and scenarios.
  arXiv  Detail & Related papers  (2025-03-18T21:21:15Z)
• TLControl: Trajectory and Language Control for Human Motion Synthesis [68.09806223962323]
  We present TLControl, a novel method for realistic human motion synthesis. It incorporates both low-level Trajectory and high-level Language semantics controls. It is practical for interactive and high-quality animation generation.
  arXiv  Detail & Related papers  (2023-11-28T18:54:16Z)
• Motion In-Betweening with Phase Manifolds [29.673541655825332]
  This paper introduces a novel data-driven motion in-betweening system to reach target poses of characters by making use of phases variables learned by a Periodic Autoencoder. Our approach utilizes a mixture-of-experts neural network model, in which the phases cluster movements in both space and time with different expert weights.
  arXiv  Detail & Related papers  (2023-08-24T12:56:39Z)
• Enhanced Fine-grained Motion Diffusion for Text-driven Human Motion Synthesis [21.57205701909026]
  We propose DiffKFC, a conditional diffusion model for text-driven motion synthesis with KeyFrames Collaborated. Our model achieves state-of-the-art performance in terms of semantic fidelity, but more importantly, is able to satisfy animator requirements through fine-grained guidance without tedious labor.
  arXiv  Detail & Related papers  (2023-05-23T07:41:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.