Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback

• URL: http://arxiv.org/abs/2603.26466v1
• Date: Fri, 27 Mar 2026 14:32:40 GMT
• Title: Adapt as You Say: Online Interactive Bimanual Skill Adaptation via Human Language Feedback
• Abstract summary: BiSAIL is a framework that enables zero-shot online adaptation of offline-learned bimanual skills through interactive language feedback.<n>BiSAIL significantly outperforms existing methods in human-in-the-loop adaptability, task generalization and cross-embodiment scalability.
• Score: 7.429591758877591
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing general-purpose robots capable of autonomously operating in human living environments requires the ability to adapt to continuously evolving task conditions. However, adapting high-dimensional coordinated bimanual skills to novel task variations at deployment remains a fundamental challenge. In this work, we present BiSAIL (Bimanual Skill Adaptation via Interactive Language), a novel framework that enables zero-shot online adaptation of offline-learned bimanual skills through interactive language feedback. The key idea of BiSAIL is to adopt a hierarchical reason-then-modulate paradigm, which first infers generalized adaptation objectives from multimodal task variations, and then adapts bimanual motions via diffusion modulation to achieve the inferred objectives. Extensive real-robot experiments across six bimanual tasks and two dual-arm platforms demonstrate that BiSAIL significantly outperforms existing methods in human-in-the-loop adaptability, task generalization and cross-embodiment scalability. This work enables the development of adaptive bimanual assistants that can be flexibly customized by non-expert users via intuitive verbal corrections. Experimental videos and code are available at https://rip4kobe.github.io/BiSAIL/.

Related papers

• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• WebXSkill: Skill Learning for Autonomous Web Agents [104.76374637691212]
  WebXSkill is a framework that bridges the gap between code-based skills and natural language guidance.<n>On WebArena and WebVoyager, WebXSkill improves task success rate by up to 9.8 and 12.9 points over the baseline.
  arXiv  Detail & Related papers  (2026-04-14T21:48:15Z)
• ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Intrinsic Adaptation [60.25542764389203]
  Agentic systems powered by Large Language Models (LLMs) have demonstrated remarkable potential in tackling complex, long-horizon tasks.<n>Existing approaches, relying on manual orchestration or runtime-based patches, often struggle with poor generalization and fragmented optimization.<n>We propose ToolSelf, a novel paradigm enabling tool-driven self-readjustment.
  arXiv  Detail & Related papers  (2026-02-08T09:27:18Z)
• SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending [79.83865372778273]
  We introduce SkillBlender, a novel hierarchical reinforcement learning framework for versatile humanoid loco-manipulation.<n>SkillBlender first pretrains goal-conditioned task-agnostic primitive skills, and then dynamically blends these skills to accomplish complex loco-manipulation tasks.<n>We also introduce SkillBench, a parallel, cross-embodiment, and diverse simulated benchmark containing three embodiments, four primitive skills, and eight challenging loco-manipulation tasks.
  arXiv  Detail & Related papers  (2025-06-11T03:24:26Z)
• Bi-LAT: Bilateral Control-Based Imitation Learning via Natural Language and Action Chunking with Transformers [16.724380665811044]
  Bi-LAT is a novel imitation learning framework that unifies bilateral control with natural language processing.<n>Bi-LAT learns to distinguish nuanced force requirements in real-world tasks.
  arXiv  Detail & Related papers  (2025-04-02T02:21:30Z)
• Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters [0.24106250158920473]
  Continual adaptation is essential for general autonomous agents.<n>Online Meta-Learned adapters (OMLA) facilitate knowledge transfer from previously learned tasks to current learning tasks.
  arXiv  Detail & Related papers  (2025-03-24T13:55:47Z)
• AutoPal: Autonomous Adaptation to Users for Personal AI Companionship [41.41280146492634]
  This paper emphasizes the necessity of autonomous adaptation in personal AI companionship.<n>We devise a hierarchical framework, AutoPal, that enables controllable and authentic adjustments to the agent's persona.<n>Experiments demonstrate the effectiveness of AutoPal and highlight the importance of autonomous adaptability in AI companionship.
  arXiv  Detail & Related papers  (2024-06-20T03:02:38Z)
• Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment [88.06408322210025]
  We study the problem of adapting on-the-fly to novel scenarios during deployment.<n>Our approach, RObust Autonomous Modulation (ROAM), introduces a mechanism based on the perceived value of pre-trained behaviors.<n>We demonstrate that ROAM enables a robot to adapt rapidly to changes in dynamics both in simulation and on a real Go1 quadruped.
  arXiv  Detail & Related papers  (2023-11-02T08:22:28Z)
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy [70.45420918526926]
  We present LILAC, a framework for incorporating and adapting to natural language corrections online during execution. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot. We show that our corrections-aware approach obtains higher task completion rates, and is subjectively preferred by users.
  arXiv  Detail & Related papers  (2023-01-06T15:03:27Z)
• Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning [69.1137074774244]
  Leveraging language interactions effectively requires addressing limitations in the two most common approaches to language grounding. We introduce the idea of neural abstructions: a set of constraints on the inference procedure of a label-conditioned generative model. We show that with this method a user population is able to build a semantic modification for an open-ended house task in Minecraft.
  arXiv  Detail & Related papers  (2021-07-20T07:01:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.