SpaceDex: Generalizable Dexterous Grasping in Tiered Workspaces

• URL: http://arxiv.org/abs/2604.17888v1
• Date: Mon, 20 Apr 2026 07:01:21 GMT
• Title: SpaceDex: Generalizable Dexterous Grasping in Tiered Workspaces
• Abstract summary: We present SpaceDex, a hierarchical framework for dexterous manipulation in constrained 3D environments.<n>At the high level, a Vision-Language Model (VLM) planner parses user intent and generates target bounding boxes for zero-shot segmentation and mask tracking.<n>At the low level, we introduce an arm-hand Feature Separation Network that decouples global trajectory control for the arm from geometry-aware grasp mode selection.
• Score: 12.014535031347926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalizable grasping with high-degree-of-freedom (DoF) dexterous hands remains challenging in tiered workspaces, where occlusion, narrow clearances, and height-dependent constraints are substantially stronger than in open tabletop scenes. Most existing methods are evaluated in relatively unoccluded settings and typically do not explicitly model the distinct control requirements of arm navigation and hand articulation under spatial constraints. We present SpaceDex, a hierarchical framework for dexterous manipulation in constrained 3D environments. At the high level, a Vision-Language Model (VLM) planner parses user intent, reasons about occlusion and height relations across multiple camera views, and generates target bounding boxes for zero-shot segmentation and mask tracking. This stage provides structured spatial guidance for downstream control instead of relying on single-view target selection. At the low level, we introduce an arm-hand Feature Separation Network that decouples global trajectory control for the arm from geometry-aware grasp mode selection for the hand, reducing feature interference between reaching and grasping objectives. The controller further integrates multi-view perception, fingertip tactile sensing, and a small set of recovery demonstrations to improve robustness to partial observability and off-nominal contacts. In 100 real-world trials involving over 30 unseen objects across four categories, SpaceDex achieves a 63.0\% success rate, compared with 39.0\% for a strong tabletop baseline. These results indicate that combining hierarchical spatial planning with arm-hand representation decoupling improves dexterous grasping performance in spatially constrained environments.

Related papers

• SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video [88.79625979053873]
  We propose a novel framework, SpaceEra, for pre-trained vision-language models (VLMs)<n>We extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference.<n>In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations.
  arXiv  Detail & Related papers  (2026-07-02T06:56:29Z)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning [99.49739831030388]
  We study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning.<n>We propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface.
  arXiv  Detail & Related papers  (2026-06-11T17:59:36Z)
• EmbodimentSemantic: A Spatial Scene-Graph Dataset and Benchmark for Vision-Language Models on Embodied Manipulation Trajectories [16.996900088118412]
  We introduce EmbodimentSemantic, a spatial scene-graph dataset and benchmark for evaluating relational grounding in embodied manipulation.<n>The dataset includes real-world manipulation observations collected with the low-cost SO101 robot arm.<n>To provide controlled validation, we also introduce a simulator-grounded LIBERO benchmark with over 60K manipulation frames and more than 120K camera-specific scene graphs across paired third-person and wrist views.
  arXiv  Detail & Related papers  (2026-06-06T18:58:54Z)
• Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints [87.13154261503168]
  Motion-controllable video generation is crucial for egocentric applications in virtual reality and embodied AI.<n>Existing methods often struggle to achieve 3D-consistent fine-grained hand articulation.<n>We propose a novel framework that generates egocentric videos from a single reference frame.
  arXiv  Detail & Related papers  (2026-03-12T10:02:23Z)
• Fast and Safe Trajectory Optimization for Mobile Manipulators With Neural Configuration Space Distance Field [34.07401865897874]
  Generalized Space Fields (GCDF) can be used to extend base-arm coupling to mobile manipulators.<n>GCDF encodes whole-body configuration in configuration in unbounded space.<n>We develop a high-performance optimization solver centered on GCDF-based reasoning.
  arXiv  Detail & Related papers  (2026-01-26T14:55:26Z)
• From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors [54.84863164684646]
  Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders.<n>In this work, we introduce FALCON, a novel paradigm that injects rich 3D spatial tokens into the action head.
  arXiv  Detail & Related papers  (2025-10-20T11:26:45Z)
• GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigation [61.34589819350429]
  We propose a training-free framework for vision-and-language navigation (VLN)<n>Our framework formulates navigation guidance as graph constraint optimization by decomposing instructions into explicit spatial constraints.<n>Our framework can effectively generalize to new environments and instruction sets, paving the way for a more robust and autonomous navigation framework.
  arXiv  Detail & Related papers  (2025-09-12T17:59:58Z)
• Graph-Guided Dual-Level Augmentation for 3D Scene Segmentation [21.553363236403822]
  3D point cloud segmentation aims to assign semantic labels to individual points in a scene for fine-grained spatial understanding.<n>Existing methods typically adopt data augmentation to alleviate the burden of large-scale annotation.<n>We propose a graph-guided data augmentation framework with dual-level constraints for realistic 3D scene synthesis.
  arXiv  Detail & Related papers  (2025-07-30T13:25:36Z)
• ReSem3D: Refinable 3D Spatial Constraints via Fine-Grained Semantic Grounding for Generalizable Robotic Manipulation [12.059517583878756]
  We propose ReSem3D, a unified manipulation framework for semantically diverse environments.<n>We show that ReSem3D performs diverse manipulation tasks under zero-shot conditions, exhibiting strong adaptability and generalization.
  arXiv  Detail & Related papers  (2025-07-24T10:07:31Z)
• RoomCraft: Controllable and Complete 3D Indoor Scene Generation [51.19602078504066]
  RoomCraft is a multi-stage pipeline that converts real images, sketches, or text descriptions into coherent 3D indoor scenes.<n>Our approach combines a scene generation pipeline with a constraint-driven optimization framework.<n>RoomCraft significantly outperforms existing methods in generating realistic, semantically coherent, and visually appealing room layouts.
  arXiv  Detail & Related papers  (2025-06-27T15:03:17Z)
• Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoders [29.274913619777088]
  We propose an occlusion-aware hand-object pose estimation method based on masked autoencoders, termed as HOMAE.<n>We integrate multi-scale features extracted from the decoder to predict a signed distance field (SDF), capturing both global context and fine-grained geometry.<n>Experiments on challenging DexYCB and HO3Dv2 benchmarks demonstrate that HOMAE achieves state-of-the-art performance in hand-object pose estimation.
  arXiv  Detail & Related papers  (2025-06-12T15:30:47Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• Reconstructing Hand-Object Interactions in the Wild [71.16013096764046]
  We propose an optimization-based procedure which does not require direct 3D supervision. We exploit all available related data (2D bounding boxes, 2D hand keypoints, 2D instance masks, 3D object models, 3D in-the-lab MoCap) to provide constraints for the 3D reconstruction. Our method produces compelling reconstructions on the challenging in-the-wild data from the EPIC Kitchens and the 100 Days of Hands datasets.
  arXiv  Detail & Related papers  (2020-12-17T18:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.