ADAPT: Agile Diffusion Action Priors for Robust and Steerable Online Text-Driven Humanoid Control

• URL: http://arxiv.org/abs/2609.00677v1
• Date: Tue, 01 Sep 2026 03:53:38 GMT
• Title: ADAPT: Agile Diffusion Action Priors for Robust and Steerable Online Text-Driven Humanoid Control
• Abstract summary: We present ADAPT, an end-to-end framework for interactive, text-conditioned humanoid whole-body control.<n> ADAPT learns a diffusion-based action prior to text-labeled humanoid state-action trajectories.<n>Experiments demonstrate robust language-grounded skill execution, smooth interactive transitions, and style-preserving downstream control.
• Score: 25.18121219548469
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present ADAPT, an end-to-end framework for interactive, text-conditioned humanoid whole-body control. Unlike dominant text-to-motion pipelines that generate kinematic motions for a separate tracker, ADAPT solves language control with an end-to-end closed-loop control framework, where the robot must continuously respond to changing commands while maintaining balance, natural motion, and smooth transitions. ADAPT learns a diffusion-based action prior from text-labeled humanoid state-action trajectories, enabling diverse motion skills to be directly executed from language commands. To improve long-horizon robustness and smooth prompt switching, we train a lightweight residual reinforcement learning policy on top of the frozen diffusion controller. We further show that the same diffusion policy can be reused as a steerable text-conditioned motion prior for downstream task adaptation. Experiments demonstrate robust language-grounded skill execution, smooth interactive transitions, and style-preserving downstream control.

Related papers

• Progressively Learning Heterogeneous Skills in a Unified Latent Space [69.64319218245176]
  HetSkills is a novel framework designed to learn heterogeneous skills within a unified latent space for physics-based character control.<n>HetSkills begins by learning a tracking skill that establishes a strong foundation in motion control and creates a shared motion decoder.<n>We introduce motion distillation to ground text-to-motion generation in language semantics and a task-guidance module that dynamically adjusts actions based on high-level language instructions.
  arXiv  Detail & Related papers  (2026-08-24T13:48:45Z)
• TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation [48.72797830886802]
  We introduce TEXEDO, a test-time scaling framework for humanoid motion generation.<n>It improves motion quality without requiring a stronger underlying generator.<n>We show it consistently improves both tracking fidelity and text alignment.
  arXiv  Detail & Related papers  (2026-06-22T08:14:35Z)
• SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-based Humanoid Control [72.5651722107621]
  SCRIPT is a scalable diffusion policy with a multi-stage training framework for language-driven physics-based humanoid control.<n>The core of SCRIPT is a Joint Action-State-Text Diffusion Transformer (JAST-DiT), which represents actions, physical states, and text as dedicated token streams.<n>To stabilize autore control, we introduce a nonlinear history conditioning mechanism, which preserves the dense recent context and samples increasingly sparse cues from long-term history.
  arXiv  Detail & Related papers  (2026-05-21T14:17:21Z)
• Before the Body Moves: Learning Anticipatory Joint Intent for Language-Conditioned Humanoid Control [7.4507678595872795]
  We propose textbfDAJI (emphDynamics-Aligned Joint Intent), a hierarchical framework that learns an anticipatory joint-intent interface between language generation and closed-loop control.<n>Experiments show that DAJI achieves strong results in anticipatory latent learning, single-instruction generation, and streaming instruction following.
  arXiv  Detail & Related papers  (2026-05-14T06:05:24Z)
• TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Control [62.93681680333618]
  TextOp is a real-time text-driven humanoid motion generation and control framework.<n>It supports streaming language commands and on-the-fly instruction modification during execution.<n>By bridging interactive motion generation with robust whole-body control, TextOp unlocks free-form intent expression.
  arXiv  Detail & Related papers  (2026-02-07T08:42:11Z)
• From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance [55.31807046722006]
  Existing language-guided humanoid pipelines are cumbersome and untrustworthy.<n>We present RoboGhost, a language-free framework that conditions humanoid policies on language-grounded motion latents.<n>We show that RoboGhost substantially reduces deployment latency, improves success rates and tracking precision, and produces smooth, semantically aligned humanoids.
  arXiv  Detail & Related papers  (2025-10-16T17:57:47Z)
• CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control [27.418288450778192]
  Motion diffusion models and Reinforcement Learning based control for physics-based simulations have complementary strengths for human motion generation. CLoSD is a text-driven RL physics-based controller, guided by diffusion generation for various tasks. CLoSD is capable of seamlessly performing a sequence of different tasks, including navigation to a goal location, striking an object with a hand or foot as specified in a text prompt, sitting down, and getting up.
  arXiv  Detail & Related papers  (2024-10-04T13:56:48Z)
• TLControl: Trajectory and Language Control for Human Motion Synthesis [68.09806223962323]
  We present TLControl, a novel method for realistic human motion synthesis. It incorporates both low-level Trajectory and high-level Language semantics controls. It is practical for interactive and high-quality animation generation.
  arXiv  Detail & Related papers  (2023-11-28T18:54:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.