StructBiHOI: Structured Articulation Modeling for Long--Horizon Bimanual Hand--Object Interaction Generation

• URL: http://arxiv.org/abs/2603.08390v2
• Date: Tue, 10 Mar 2026 09:02:11 GMT
• Title: StructBiHOI: Structured Articulation Modeling for Long--Horizon Bimanual Hand--Object Interaction Generation
• Abstract summary: Long--horizon planning instability, fine--grained joint articulation, and complex cross--hand coordination make coherent bimanual generation difficult.<n>We propose StructBiHOI, a Structured articulation modeling framework for long-horizon Bimanual HOI generation.
• Score: 30.285126708457174
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent progress in 3D hand--object interaction (HOI) generation has primarily focused on single--hand grasp synthesis, while bimanual manipulation remains significantly more challenging. Long--horizon planning instability, fine--grained joint articulation, and complex cross--hand coordination make coherent bimanual generation difficult, especially under multimodal conditions. Existing approaches often struggle to simultaneously ensure temporal consistency, physical plausibility, and semantic alignment over extended sequences. We propose StructBiHOI, a Structured articulation modeling framework for long-horizon Bimanual HOI generation. Our key insight is to structurally disentangle temporal joint planning from frame--level manipulation refinement. Specifically, a jointVAE models long-term joint evolution conditioned on object geometry and task semantics, while a maniVAE refines fine-grained hand poses at the single--frame level. To enable stable and efficient long--sequence generation, we incorporate a state--space--inspired diffusion denoiser based on Mamba, which models long--range dependencies with linear complexity. This hierarchical design facilitates coherent dual-hand coordination and articulated object interaction. Extensive experiments on bimanual manipulation and single-hand grasping benchmarks demonstrate that our method achieves superior long--horizon stability, motion realism, and computational efficiency compared to strong baselines.

Related papers

• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• IS-CoT: Breaking the Long-form Generation Collapse via Interleaved Structural Thinking [50.381430711141995]
  We introduce the Interleaved Structural Chain-of-Thought (IS-CoT) framework.<n>IS-CoT embeds a dynamic Plan-Write-Reflect cycle into the generation process.<n>We construct a high-quality dataset of interleaved reasoning traces via a multi-teacher pipeline and train IS-Writer-8B.
  arXiv  Detail & Related papers  (2026-06-08T16:31:00Z)
• Libra-VLA: Achieving Learning Equilibrium via Asynchronous Coarse-to-Fine Dual-System [15.427164345561417]
  Vision-Language-Action (VLA) models are a promising paradigm for generalist robotic manipulation.<n>We introduce Libra-VLA, a novel Coarse-to-Fine Dual-System VLA architecture.<n>We show that our approach offers a scalable, robust, and responsive solution for open-world manipulation.
  arXiv  Detail & Related papers  (2026-04-27T19:02:46Z)
• Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation [25.677744104220853]
  Video frame aims to synthesize realistic intermediate frames between given endpoints while adhering to specific motion semantics.<n>We propose a novel bidirectional framework that enforces symmetry between forward and backward generation trajectories.<n>Our method achieves state-of-the-art performance in imaging quality, motion smoothness, and dynamic control on both 37-frame and 73-frame tasks.
  arXiv  Detail & Related papers  (2026-04-02T06:58:46Z)
• TopoOR: A Unified Topological Scene Representation for the Operating Room [54.50897207203292]
  Surgical Scene Graphs abstract the complexity of surgical operating rooms (OR) into a structure of entities and their relations.<n>We introduce TopoOR, a new paradigm that models multimodal operating rooms as a higher-order structure.<n>We also propose a higher-order attention mechanism that explicitly preserves manifold structure and modality-specific features.
  arXiv  Detail & Related papers  (2026-03-10T10:19:42Z)
• Structural Action Transformer for 3D Dexterous Manipulation [80.07649565189035]
  Cross-embodiment skill transfer is a challenge for high-DoF robotic hands.<n>Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations.<n>This paper proposes a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective.
  arXiv  Detail & Related papers  (2026-03-04T11:38:12Z)
• HINT: Hierarchical Interaction Modeling for Autoregressive Multi-Human Motion Generation [55.73037290387896]
  We introduce HINT, the first autoregressive framework for multi-human motion generation with Hierarchical INTeraction modeling in diffusion.<n>First, HINT leverages a disentangled motion representation within a canonicalized latent space, decoupling local motion semantics from inter-person interactions.<n>Second, HINT adopts a sliding-window strategy for efficient online generation, and aggregates local within-window and global cross-window conditions to capture past human history, inter-person dependencies, and align with text guidance.
  arXiv  Detail & Related papers  (2026-01-28T08:47:23Z)
• SasMamba: A Lightweight Structure-Aware Stride State Space Model for 3D Human Pose Estimation [0.8427427828815586]
  We propose a structure-awaretemporal convolution to dynamically capture essential local interactions between joints.<n>We then apply a stride-based scan strategy to construct multi-scale global structural representations.<n>Our model SasMamba achieves competitive 3D pose estimation performance with significantly fewer parameters compared to existing hybrid models.
  arXiv  Detail & Related papers  (2025-11-12T01:25:20Z)
• Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts [10.204413386807564]
  We propose a Dynamic Adaptive Shared Expert and Grouped Multi-Head Attention Hybrid Model (DASG-MoE) to enhance long-sequence modeling capabilities.<n>First, we employ the Grouped Multi-Head Attention (GMHA) mechanism to effectively reduce the computational complexity of long sequences.<n>Second, we design a Dual-Scale Shared Expert Structure (DSSE), where shallow experts use lightweight computations to quickly respond to low-dimensional features.<n>Third, we propose a hierarchical Adaptive Dynamic Routing (ADR) mechanism that dynamically selects expert levels based on feature complexity and task requirements.
  arXiv  Detail & Related papers  (2025-09-05T02:49:15Z)
• Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance [61.41904916189093]
  We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images.<n>We use hand-object interaction as geometric guidance to ensure plausible hand-object interactions.
  arXiv  Detail & Related papers  (2025-08-25T17:11:53Z)
• Towards Consistent Long-Term Pose Generation [0.0]
  We propose a novel one-stage architecture that directly generates poses in continuous coordinate space from minimal context.<n>Our key innovation is eliminating the need for intermediate representations or token-based generation.<n>Our approach significantly outperforms existing quantization-based and autoregressive methods, especially in long-term generation scenarios.
  arXiv  Detail & Related papers  (2025-07-24T12:57:22Z)
• Learning to Align and Refine: A Foundation-to-Diffusion Framework for Occlusion-Robust Two-Hand Reconstruction [50.952228546326516]
  Two-hand reconstruction from monocular images faces persistent challenges due to complex and dynamic hand postures.<n>Existing approaches struggle with such alignment issues, often resulting in misalignment and penetration artifacts.<n>We propose a dual-stage Foundation-to-Diffusion framework that precisely align 2D prior guidance from vision foundation models.
  arXiv  Detail & Related papers  (2025-03-22T14:42:27Z)
• FastGrasp: Efficient Grasp Synthesis with Diffusion [25.91329341243337]
  We introduce a novel diffusion-model-based approach that generates the grasping pose in a one-stage manner. This allows us to significantly improve generation speed and the diversity of generated hand poses. Our method achieves faster inference, higher diversity, and superior pose quality than state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-11-22T08:06:32Z)
• Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling [4.190836962132713]
  This paper introduces Orchid, a novel architecture designed to address the quadratic complexity of traditional attention mechanisms. At the core of this architecture lies a new data-dependent global convolution layer, which contextually adapts its conditioned kernel on input sequence. We evaluate the proposed model across multiple domains, including language modeling and image classification, to highlight its performance and generality.
  arXiv  Detail & Related papers  (2024-02-28T17:36:45Z)
• Generative Hierarchical Temporal Transformer for Hand Pose and Action Modeling [67.94143911629143]
  We propose a generative Transformer VAE architecture to model hand pose and action. To faithfully model the semantic dependency and different temporal granularity of hand pose and action, we decompose the framework into two cascaded VAE blocks. Results show that our joint modeling of recognition and prediction improves over isolated solutions.
  arXiv  Detail & Related papers  (2023-11-29T05:28:39Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.