Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory for Long-Horizon Contact-Rich Robotic Manipulation

• URL: http://arxiv.org/abs/2603.13315v1
• Date: Wed, 04 Mar 2026 09:16:30 GMT
• Title: Bi-HIL: Bilateral Control-Based Multimodal Hierarchical Imitation Learning via Subtask-Level Progress Rate and Keyframe Memory for Long-Horizon Contact-Rich Robotic Manipulation
• Abstract summary: Long-horizon contact-rich robotic manipulation is challenging due to observability and unstable subtask transitions.<n>We propose Bi-HIL, a bilateral control-based multimodal hierarchical imitation learning framework for long-horizon manipulation.
• Score: 1.552282932199974
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon contact-rich robotic manipulation remains challenging due to partial observability and unstable subtask transitions under contact uncertainty. While hierarchical architectures improve temporal reasoning and bilateral imitation learning enables force-aware control, existing approaches often rely on flat policies that struggle with long-horizon coordination. We propose Bi-HIL, a bilateral control-based multimodal hierarchical imitation learning framework for long-horizon manipulation. Bi-HIL stabilizes hierarchical coordination by integrating keyframe memory with subtask-level progress rate that models phase progression within the active subtask and conditions both high- and low-level policies. We evaluate Bi-HIL on unimanual and bimanual real-robot tasks, demonstrating consistent improvements over flat and ablated variants. The results highlight the importance of explicitly modeling subtask progression together with force-aware control for robust long-horizon manipulation. For additional material, please check: https://mertcookimg.github.io/bi-hil

Related papers

• $τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation [51.05516457377339]
  Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks.<n>We introduce $_0$-VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem.<n>A low-level policy then executes the generated subtask across multiple robot embodiments.
  arXiv  Detail & Related papers  (2026-08-17T17:59:11Z)
• Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control [21.97625864571452]
  Language-conditioned Imitation Learning (IL) is essential for enabling robots to perform complex tasks following natural language instructions.<n>We propose Dual-Process Atomic Skill Learning (DASL), a novel hierarchical imitation learning framework that decouples slow semantic reasoning from real-time motion control.<n>DASL significantly outperforms state-of-the-art baselines, excelling in skill acquisition and compositional generalization to unseen instructions.
  arXiv  Detail & Related papers  (2026-07-12T07:38:53Z)
• BiCoord: A Bimanual Manipulation Benchmark towards Long-Horizon Spatial-Temporal Coordination [20.953323366922266]
  BiCoord is a benchmark for long-horizon and tightly coordinated bimanual manipulation.<n>Specifically, BiCoord comprises diverse tasks that require continuous inter-arm dependency and dynamic role exchange.<n>Results show that representative manipulation policies struggle with long-duration and highly coupled tasks.
  arXiv  Detail & Related papers  (2026-04-07T13:02:17Z)
• StructBiHOI: Structured Articulation Modeling for Long--Horizon Bimanual Hand--Object Interaction Generation [30.285126708457174]
  Long--horizon planning instability, fine--grained joint articulation, and complex cross--hand coordination make coherent bimanual generation difficult.<n>We propose StructBiHOI, a Structured articulation modeling framework for long-horizon Bimanual HOI generation.
  arXiv  Detail & Related papers  (2026-03-09T13:47:06Z)
• Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport [36.304887250687536]
  This paper introduces cognition-to-control (C2C), a three-layer hierarchy that makes the deliberation-to-control pathway explicit.<n> Experiments on collaborative manipulation tasks show higher success and robustness than single-agent and end-to-end baselines.
  arXiv  Detail & Related papers  (2026-03-04T06:24:55Z)
• Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining [56.62125584296097]
  Keyframe-Chaining VLA is a framework that extracts and links key historical frames to model long-horizon dependencies.<n>We design a progress-aware mechanism that dynamically retrieves historical frames based on their temporal relevance to the current execution phase.<n>We introduce a suite of four Non-Markovian manipulation tasks built upon the ManiSkill simulator to measure task success rates.
  arXiv  Detail & Related papers  (2026-03-02T05:26:29Z)
• BiManiBench: A Hierarchical Benchmark for Evaluating Bimanual Coordination of Multimodal Large Language Models [30.38546017617527]
  We introduce BiManiBench, a benchmark evaluating Multi Large Language Models (MLLMs) across three tiers.<n>Our framework isolates unique bimanual challenges, such as arm reachability and kinematic constraints.<n>Analysis of over 30 state-of-the-art models reveals that despite high-level reasoning proficiency, MLLMs struggle with dual-arm spatial grounding and control.
  arXiv  Detail & Related papers  (2026-02-09T08:47:14Z)
• Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering [59.18634614089481]
  We present ML-Master 2.0, an autonomous agent that masters ultra-long-horizon machine learning engineering (MLE)<n>By reframing context management as a process of cognitive accumulation, our approach introduces Hierarchical Cognitive Caching (HCC)<n>HCC allows agents to decouple immediate execution from long-term experimental strategy.<n>In evaluations on OpenAI's MLE-Bench under 24-hour budgets, ML-Master 2.0 achieves a state-of-the-art medal rate of 56.44%.
  arXiv  Detail & Related papers  (2026-01-15T13:52:04Z)
• Heterogeneous Multi-Expert Reinforcement Learning for Long-Horizon Multi-Goal Tasks in Autonomous Forklifts [5.215925647203835]
  We propose a Heterogeneous Multi-Expert Reinforcement Learning (HMER) framework tailored for autonomous forklifts.<n>HMER decomposes long-horizon tasks into specialized sub-policies controlled by a Semantic Task Planner.<n>Our method achieves a task success rate of 94.2% (compared to 62.5% for baselines), reduces operation time by 21.4%, and maintains placement error within 1.5 cm.
  arXiv  Detail & Related papers  (2026-01-12T08:27:24Z)
• PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation [27.791908160098625]
  PALM builds policy learning around interaction-centric affordance reasoning and subtask progress cues.<n>Palm consistently outperforms baselines in simulation and real-world experiments.
  arXiv  Detail & Related papers  (2026-01-11T21:00:58Z)
• RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation [59.9896841079005]
  We introduce RoboCerebra, a benchmark for evaluating high-level reasoning in long-horizon robotic manipulation.<n>The dataset is constructed via a top-down pipeline, where GPT generates task instructions and decomposes them into subtask sequences.<n>Compared to prior benchmarks, RoboCerebra features significantly longer action sequences and denser annotations.
  arXiv  Detail & Related papers  (2025-06-07T06:15:49Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)
• Modeling Long-horizon Tasks as Sequential Interaction Landscapes [75.5824586200507]
  We present a deep learning network that learns dependencies and transitions across subtasks solely from a set of demonstration videos. We show that these symbols can be learned and predicted directly from image observations. We evaluate our framework on two long horizon tasks: (1) block stacking of puzzle pieces being executed by humans, and (2) a robot manipulation task involving pick and place of objects and sliding a cabinet door with a 7-DoF robot arm.
  arXiv  Detail & Related papers  (2020-06-08T18:07:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.