See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation

• URL: http://arxiv.org/abs/2606.13279v1
• Date: Thu, 11 Jun 2026 12:33:55 GMT
• Title: See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation
• Abstract summary: We propose a bimanual manipulation framework based on Dual-Level Structural Decomposition.<n>We evaluate the proposed method on six simulated bimanual manipulation tasks in RoboTwin 2.0 and three long-horizon real-world tasks.
• Score: 1.7587442088965224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In bimanual robotic manipulation, task-relevant visual information varies with the task stage and context, while the interaction of the two arms shifts between independent and coordinated modes, making policy learning challenging. However, existing monolithic Vision-Language-Action (VLA) policies process diverse visual inputs and interaction patterns through a single shared representation and action generation pathway, often failing to separately account for visual relevance and bimanual interaction structure. To address this issue, we propose a bimanual manipulation VLA framework based on Dual-Level Structural Decomposition. The View-Selective Visual Router dynamically adjusts wrist-view contributions to emphasize relevant visual cues, while the Interaction-Aware Action Mixture-of-Experts (MoE) decomposes action generation into coordinated and arm-wise pathways to adapt to varying bimanual interaction modes. We evaluate the proposed method on six simulated bimanual manipulation tasks in RoboTwin 2.0 and three long-horizon real-world tasks. Our model improves the overall average success rate over a monolithic baseline by 27.7% in simulation and 43.3% in real-world evaluation, while consistently outperforming single-module variants across both settings. These results demonstrate that jointly considering selective visual processing and explicit decomposition of bimanual interaction structures provides an effective inductive bias for robust bimanual manipulation.

Related papers

• AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning [52.47988016454267]
  Vision-Language-Action (VLA) models excel at end-to-end robotic manipulation but struggle with out-of-distribution (OOD) generalization.<n>We identify two mutually reinforcing modes: emphtrajectory overfitting, where models overfit to holistic trajectory patterns rather than compositional sub-skill semantics; and emphperceptual shortcut, where action tokens over-rely on wrist-view textures at the expense of global spatial grounding.<n>We introduce textbfAC-VLA, a plug-and-play Action Compositional learning
  arXiv  Detail & Related papers  (2026-07-17T07:51:03Z)
• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations [19.347332945780273]
  Vision-Language-Action (VLA) models have shown strong potential for general-purpose robot manipulation by unifying perception and action.<n>We introduce gesture as a parallel instruction modality and propose a Gesture-aware Vision-Language-Action model (GesVLA)<n> Experimental results show that incorporating gesture consistently improves target grounding accuracy and human-robot interaction efficiency.
  arXiv  Detail & Related papers  (2026-05-21T17:57:44Z)
• TriRelVLA: Triadic Relational Structure for Generalizable Embodied Manipulation [54.81449795163812]
  Vision-language--action (VLA) models perform well on training-seen robotic tasks but struggle to generalize to unseen scenes and objects.<n>We propose TriRelVLA, a triadic relational VLA framework for generalizable embodied manipulation.<n> Experiments show strong performance on fine-tuned tasks and clear gains in cross-scene, cross-object, and cross-task generalization.
  arXiv  Detail & Related papers  (2026-05-07T05:57:49Z)
• Scene-agnostic Hierarchical Bimanual Task Planning via Visual Affordance Reasoning [5.296623863894165]
  Embodied agents operating in open environments must translate high-level instructions into grounded, executable behaviors.<n>We present a unified framework for scene-agnostic bimanual task planning that bridges high-level reasoning with 3D-grounded two-handed execution.
  arXiv  Detail & Related papers  (2025-12-10T04:37:07Z)
• Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework [51.39847596489193]
  Bimanual robotic manipulation is an emerging and critical topic in the robotics community.<n>We propose a novel decoupled interaction framework that considers the characteristics of different tasks in bimanual manipulation.<n>Our framework achieves outstanding performance, with a 23.5% boost over the SOTA method.
  arXiv  Detail & Related papers  (2025-03-12T09:28:41Z)
• InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation [1.4835219023165949]
  We introduce InterACT: Inter-dependency aware Action Chunking with Hierarchical Attention Transformers. InterACT is a novel imitation learning framework designed specifically for bimanual manipulation. Our experiments, conducted on various simulated and real-world bimanual manipulation tasks, demonstrate that InterACT outperforms existing methods.
  arXiv  Detail & Related papers  (2024-09-12T10:30:44Z)
• Learning Manipulation by Predicting Interaction [85.57297574510507]
  We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction. The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
  arXiv  Detail & Related papers  (2024-06-01T13:28:31Z)
• Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection [116.21529970404653]
  We introduce SG2HOI+, a unified one-step model based on the Transformer architecture. Our approach employs two interactive hierarchical Transformers to seamlessly unify the tasks of SGG and HOI detection. Our approach achieves competitive performance when compared to state-of-the-art HOI methods.
  arXiv  Detail & Related papers  (2023-11-03T07:25:57Z)
• Multi-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimation [49.56131393810713]
  We present an SE(3) equivariant architecture and a training strategy to tackle this task in an unsupervised manner. Our method excels in both model performance and computational efficiency, with only 0.25M parameters and 0.92G FLOPs.
  arXiv  Detail & Related papers  (2023-06-08T22:55:32Z)
• Task-Aware Asynchronous Multi-Task Model with Class Incremental Contrastive Learning for Surgical Scene Understanding [17.80234074699157]
  A multi-task learning model is proposed for surgical report generation and tool-tissue interaction prediction. The model forms of shared feature extractor, mesh-transformer branch for captioning and graph attention branch for tool-tissue interaction prediction. We incorporate a task-aware asynchronous MTL optimization technique to fine-tune the shared weights and converge both tasks optimally.
  arXiv  Detail & Related papers  (2022-11-28T14:08:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.