VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning

• URL: http://arxiv.org/abs/2603.17720v1
• Date: Wed, 18 Mar 2026 13:40:24 GMT
• Title: VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning
• Abstract summary: VolumeDP is a policy architecture that restores spatial alignment by explicitly reasoning in 3D.<n>It achieves a state-of-the-art average success rate of 88.8% on the LIBERO simulation benchmark.
• Score: 29.62906091681386
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning is a prominent paradigm for robotic manipulation. However, existing visual imitation methods map 2D image observations directly to 3D action outputs, imposing a 2D-3D mismatch that hinders spatial reasoning and degrades robustness. We present VolumeDP, a policy architecture that restores spatial alignment by explicitly reasoning in 3D. VolumeDP first lifts image features into a Volumetric Representation via cross-attention. It then selects task-relevant voxels with a learnable module and converts them into a compact set of spatial tokens, markedly reducing computation while preserving action-critical geometry. Finally, a multi-token decoder conditions on the entire token set to predict actions, thereby avoiding lossy aggregation that collapses multiple spatial tokens into a single descriptor. VolumeDP achieves a state-of-the-art average success rate of 88.8% on the LIBERO simulation benchmark, outperforming the strongest baseline by a substantial 14.8% improvement. It also delivers large performance gains over prior methods on the ManiSkill and LIBERO-Plus benchmarks. Real-world experiments further demonstrate higher success rates and robust generalization to novel spatial layouts, camera viewpoints, and environment backgrounds. Code will be released.

Related papers

• Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification [22.850651559843104]
  Action Map Policy casts 3D closed-loop manipulation policy learning as a classification problem in image space.<n>We show that AMP outperforms strong baselines, achieving higher success rates, faster inference, and enhanced spatial reasoning.
  arXiv  Detail & Related papers  (2026-07-12T10:58:06Z)
• Semantic-Guided Progressive Object Removal with Gaussian Splatting [8.821363895807243]
  We propose a novel framework that integrates Semantic-guided Block Matching (SBM) and Region-Wise Progressive Refinement (RPR) for high-quality 3D object removal.<n> Experimental results demonstrate that our approach outperforms existing Gaussian-based methods in terms of perceptual quality and coherence in 3D object removal.
  arXiv  Detail & Related papers  (2026-07-05T07:11:48Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction [57.63073414949329]
  We propose PointACT, a dual-system 3D-aware VLA policy that integrates hierarchical 3D point cloud representations directly into the action decoding process.<n>PointACT employs a multi-scale point-action interaction mechanism with efficient bottleneck window self-attention, enabling evolving action tokens to densely attend to both local geometric detail and global scene structure.
  arXiv  Detail & Related papers  (2026-05-20T17:10:31Z)
• Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation [67.16145181790522]
  This paper tackles spatial perception and manipulation challenges in Vision-Language-Action (VLA) models.<n>We propose a Geometry-Guided Gated Transformer (G3T) that aligns multi-view features under 3D geometric guidance.<n>To improve action learning efficiency, we introduce Action Manifold Learning (AML), which directly predicts actions on the valid action manifold.
  arXiv  Detail & Related papers  (2026-05-12T09:21:29Z)
• LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windows [10.300202521638274]
  We introduce the Large Sparse Reconstruction Model to study how scaling transformer context windows impacts feed-forward 3D reconstruction.<n>We show that expanding the context window -- by substantially increasing the number of active object and image tokens -- remarkably narrows this gap and enables high-fidelity 3D object reconstruction and inverse rendering.
  arXiv  Detail & Related papers  (2026-04-06T21:21:12Z)
• Action-Geometry Prediction with 3D Geometric Prior for Bimanual Manipulation [53.09168514034483]
  Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions.<n>We propose a framework that builds bimanual manipulation directly on a pre-trained 3D geometric foundation model.<n>Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap.
  arXiv  Detail & Related papers  (2026-02-27T08:54:20Z)
• SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation [63.48859753472547]
  SpatialActor is a framework for robust robotic manipulation that explicitly decouples semantics and geometry.<n>It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions.
  arXiv  Detail & Related papers  (2025-11-12T18:59:08Z)
• From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors [54.84863164684646]
  Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders.<n>In this work, we introduce FALCON, a novel paradigm that injects rich 3D spatial tokens into the action head.
  arXiv  Detail & Related papers  (2025-10-20T11:26:45Z)
• Gaussian2Scene: 3D Scene Representation Learning via Self-supervised Learning with 3D Gaussian Splatting [6.678115792482272]
  Self-supervised learning (SSL) for point cloud pre-training has become a cornerstone for many 3D vision tasks.<n>We propose a novel scene-level SSL framework that leverages the efficiency and explicit nature of 3D Gaussian Splatting (3DGS) for pre-training.
  arXiv  Detail & Related papers  (2025-06-10T13:19:21Z)
• HiPART: Hierarchical Pose AutoRegressive Transformer for Occluded 3D Human Pose Estimation [61.32714172038278]
  We propose a novel two-stage generative densification method, named Hierarchical Pose AutoRegressive Transformer (HiPART), to generate 2D dense poses from the original sparse 2D pose.<n>Specifically, we first develop a multi-scale skeleton tokenization module to quantize the highly dense 2D pose into hierarchical tokens and propose a Skeleton-aware Alignment to strengthen token connections.<n>With generated hierarchical poses as inputs for 2D-to-3D lifting, the proposed method shows strong robustness in occluded scenarios and achieves state-of-the-art performance on the single-frame-based 3D
  arXiv  Detail & Related papers  (2025-03-30T06:15:36Z)
• FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding [11.118857208538039]
  We present Foundation Model Embedded Gaussian Splatting (S), which incorporates vision-language embeddings of foundation models into 3D Gaussian Splatting (GS) Results demonstrate remarkable multi-view semantic consistency, facilitating diverse downstream tasks, beating state-of-the-art methods by 10.2 percent on open-vocabulary language-based object detection. This research explores the intersection of vision, language, and 3D scene representation, paving the way for enhanced scene understanding in uncontrolled real-world environments.
  arXiv  Detail & Related papers  (2024-01-03T20:39:02Z)
• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [111.16358607889609]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation.<n>For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulation [18.964403296437027]
  Act3D represents the robot's workspace using a 3D feature field with adaptive resolutions dependent on the task at hand. It samples 3D point grids in a coarse to fine manner, featurizes them using relative-position attention, and selects where to focus the next round of point sampling.
  arXiv  Detail & Related papers  (2023-06-30T17:34:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.