Information-Theoretic Detection of Bimanual Interactions for Dual-Arm Robot Plan Generation

• URL: http://arxiv.org/abs/2601.19832v1
• Date: Tue, 27 Jan 2026 17:38:58 GMT
• Title: Information-Theoretic Detection of Bimanual Interactions for Dual-Arm Robot Plan Generation
• Abstract summary: This paper presents a novel one-shot method for processing a single RGB video of a bimanual task demonstration to generate an execution plan for a dual-arm robotic system.<n>We apply Shannon's information theory to analyze the information flow between scene elements and leverage scene graph properties.<n>The generated plan is a modular behavior tree that assumes different structures based on the desired arms coordination.
• Score: 14.053079591358797
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Programming by demonstration is a strategy to simplify the robot programming process for non-experts via human demonstrations. However, its adoption for bimanual tasks is an underexplored problem due to the complexity of hand coordination, which also hinders data recording. This paper presents a novel one-shot method for processing a single RGB video of a bimanual task demonstration to generate an execution plan for a dual-arm robotic system. To detect hand coordination policies, we apply Shannon's information theory to analyze the information flow between scene elements and leverage scene graph properties. The generated plan is a modular behavior tree that assumes different structures based on the desired arms coordination. We validated the effectiveness of this framework through multiple subject video demonstrations, which we collected and made open-source, and exploiting data from an external, publicly available dataset. Comparisons with existing methods revealed significant improvements in generating a centralized execution plan for coordinating two-arm systems.

Related papers

• See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation [1.7587442088965224]
  We propose a bimanual manipulation framework based on Dual-Level Structural Decomposition.<n>We evaluate the proposed method on six simulated bimanual manipulation tasks in RoboTwin 2.0 and three long-horizon real-world tasks.
  arXiv  Detail & Related papers  (2026-06-11T12:33:55Z)
• Learning Causal Structure Distributions for Robust Planning [53.753366558072806]
  We find that learning the functional relationships while accounting for the uncertainty about the structural information leads to more robust dynamics models.<n>This in contrast with common model-learning methods that ignore the causal structure and fail to leverage the sparsity of interactions in robotic systems.<n>We show that our model can be used to learn the dynamics of a robot, which together with a sampling-based planner can be used to perform new tasks in novel environments.
  arXiv  Detail & Related papers  (2025-08-08T22:43:17Z)
• Information-Theoretic Graph Fusion with Vision-Language-Action Model for Policy Reasoning and Dual Robotic Control [22.74768543283102]
  Graph-Fused Vision-Language-Action (GF-VLA) is a framework that enables dual-arm robotic systems to perform task-level reasoning and execution.<n>GF-VLA first extracts Shannon-information-based cues to identify hands and objects with the highest task relevance.<n>Cross-hand selection policy infers optimal assignment without explicit geometric reasoning.
  arXiv  Detail & Related papers  (2025-08-07T12:48:09Z)
• Unifying Search and Recommendation with Dual-View Representation Learning in a Generative Paradigm [51.2624255871896]
  GenSR is a novel generative paradigm for unifying search and recommendation.<n>Our work introduces a new generative paradigm compared with previous discriminative methods.
  arXiv  Detail & Related papers  (2025-04-09T09:15:37Z)
• Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework [51.39847596489193]
  Bimanual robotic manipulation is an emerging and critical topic in the robotics community.<n>We propose a novel decoupled interaction framework that considers the characteristics of different tasks in bimanual manipulation.<n>Our framework achieves outstanding performance, with a 23.5% boost over the SOTA method.
  arXiv  Detail & Related papers  (2025-03-12T09:28:41Z)
• Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation [16.244250979166214]
  Generalizable Planning-Guided Diffusion Policy Learning (GLIDE) is an approach that learns to solve contact-rich bimanual manipulation tasks.<n>We propose a set of essential design options in feature extraction, task representation, action prediction, and data augmentation.<n>Our approach can enable a bimanual robotic system to effectively manipulate objects of diverse geometries, dimensions, and physical properties.
  arXiv  Detail & Related papers  (2024-12-03T18:51:39Z)
• InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation [1.4835219023165949]
  We introduce InterACT: Inter-dependency aware Action Chunking with Hierarchical Attention Transformers. InterACT is a novel imitation learning framework designed specifically for bimanual manipulation. Our experiments, conducted on various simulated and real-world bimanual manipulation tasks, demonstrate that InterACT outperforms existing methods.
  arXiv  Detail & Related papers  (2024-09-12T10:30:44Z)
• Controlled Training Data Generation with Diffusion Models [48.123126522294015]
  We present a method to control a text-to-image generative model to produce training data useful for supervised learning.<n>We develop an automated closed-loop system which involves two feedback mechanisms.
  arXiv  Detail & Related papers  (2024-03-22T15:59:24Z)
• Unsupervised Task Graph Generation from Instructional Video Transcripts [53.54435048879365]
  We consider a setting where text transcripts of instructional videos performing a real-world activity are provided. The goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel task graph generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components.
  arXiv  Detail & Related papers  (2023-02-17T22:50:08Z)
• Process Discovery for Structured Program Synthesis [70.29027202357385]
  A core task in process mining is process discovery which aims to learn an accurate process model from event log data. In this paper, we propose to use (block-) structured programs directly as target process models. We develop a novel bottom-up agglomerative approach to the discovery of such structured program process models.
  arXiv  Detail & Related papers  (2020-08-13T10:33:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.