GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors

• URL: http://arxiv.org/abs/2606.08440v1
• Date: Sun, 07 Jun 2026 03:37:55 GMT
• Title: GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors
• Abstract summary: We present GraspFoM, a unified framework that builds a shared 3D object latent for both reconstruction and grasp pose prediction.<n>We show that GraspFoM achieves state-of-the-art results on both reconstruction and grasping.
• Score: 23.311035802788854
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robotic grasping is a fundamental capability in robotic manipulation. Yet grasping remains challenging under partial observations. Reliable grasping depends on both local contact cues and object-level 3D structure. Existing geometry-aware grasping methods recognize the value of reconstruction, but they typically treat geometry as an intermediate prediction rather than a reusable object prior for grasping. In this paper, we present GraspFoM, a unified framework that leverages 3D foundation priors (SAM3D) to build a shared 3D object latent for both reconstruction and grasp pose prediction. Built on this shared object latent, we introduce an anchor-initialized truncated pose-reasoning diffuser that predicts continuous and multimodal grasp poses without directly relying on discrete grasp candidates. We further investigate the interaction between reconstruction and grasping through a reconstruction-aware scorer and a residual latent updater. Reconstruction provides grounded geometric cues, while grasp supervision refines the shared object latent toward grasp-relevant affordances. GraspFoM jointly predicts grasp poses and reconstructs high-fidelity 3D assets in mesh and 3DGS forms. Comprehensive experiments demonstrate that GraspFoM achieves state-of-the-art results on both reconstruction and grasping. Notably, these improvements require only a small number of additional trainable parameters. Component-wise ablation studies also demonstrate the contribution of each component.

Related papers

• Articulated Object Reconstruction from Rest-State Observation [33.09154459685742]
  Building interactive digital twins requires recovering both 3D geometry and the kinematic structures that govern how objects articulate.<n>We introduce a rest-state formulation that reconstructs articulated objects from a single closed configuration.
  arXiv  Detail & Related papers  (2026-07-30T06:42:03Z)
• Artic-O: End-to-End Articulated Object Reconstruction via Latent Geometry Learning [105.35529379360186]
  Artic-O is an end-to-end, feed-forward framework for articulated object reconstruction.<n>It maps sparse multi-state observations into a pretrained latent geometry space.<n>A frozen flow-matching decoder provides a complete-shape prior for recovering visible and occluded structures.
  arXiv  Detail & Related papers  (2026-06-20T08:15:30Z)
• Pano3D: Unified 3D Reconstruction and Panoptic Segmentation [68.18993232814061]
  We introduce an approach that performs 3D reconstruction and 3D panoptic segmentation in a unified framework.<n>We build on existing 3D reconstruction models and augment them with a set-based mask decoder.<n>Our method achieves state-of-the-art performance in 3D panoptic segmentation across ScanNet, ScanNet200, and ScanNet++ datasets.
  arXiv  Detail & Related papers  (2026-06-12T09:43:13Z)
• GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images [61.2170105836525]
  We propose an RGB-only framework that reformulates reconstruction as physically-grounded scene factorization and outputs a structured hybrid scene representation.<n>The resulting representation combines explicit rigid bodies with a decoupled background, enabling direct physics simulation while preserving visual realism.
  arXiv  Detail & Related papers  (2026-06-02T17:13:01Z)
• Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch [3.926587614210279]
  We propose a multimodal, physically grounded approach for metric-scale amodal object reconstruction and pose estimation under hand occlusion.<n>We leverage physical interaction signals: proprioception provides the posed hand geometry, and multi-contact touch constrains where the object surface must lie.
  arXiv  Detail & Related papers  (2026-04-10T08:32:51Z)
• Reliev3R: Relieving Feed-forward Reconstruction from Multi-View Geometric Annotations [98.66466590444553]
  We propose Reliev3R, a weakly-supervised paradigm for training FFRMs from scratch without cost-prohibitive multi-view geometric annotations.<n>At the core of Reliev3R, we design an ambiguity-aware relative depth loss and a trigonometry-based reprojection loss to facilitate supervision for multi-view geometric consistency.
  arXiv  Detail & Related papers  (2026-04-01T06:46:54Z)
• Interact3D: Compositional 3D Generation of Interactive Objects [31.12099147294145]
  We present a novel framework designed to generate physically plausible interacting 3D compositional objects.<n>Our approach first leverages advanced generative priors to curate high-quality individual assets.<n>To physically compose these assets, we then introduce a robust two-stage composition pipeline.
  arXiv  Detail & Related papers  (2026-03-17T03:21:06Z)
• Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery [60.51998732898099]
  We introduce HMRMamba, a new paradigm for 3D Human Mesh Recovery.<n>It pioneers the use of Structured State Space Models for their efficiency and long-range modeling prowess.<n>Our framework is distinguished by two core contributions. First, the Geometry-Aware Lifting Module, featuring a novel dual-scan Mamba architecture.
  arXiv  Detail & Related papers  (2026-01-29T08:05:02Z)
• Particulate: Feed-Forward 3D Object Articulation [89.78788418174946]
  Particulate is a feed-forward approach that, given a single static 3D mesh of an everyday object, directly infers all attributes of the underlying articulated structure.<n>We train the network end-to-end on a diverse collection of articulated 3D assets from public datasets.<n>During inference, Particulate lifts the network's feed-forward prediction to the input mesh, yielding a fully articulated 3D model in seconds.
  arXiv  Detail & Related papers  (2025-12-12T18:59:51Z)
• AREA3D: Active Reconstruction Agent with Unified Feed-Forward 3D Perception and Vision-Language Guidance [36.125573065910594]
  Active 3D reconstruction enables an agent to autonomously select viewpoints to obtain accurate and complete scene geometry.<n>We propose AREA3D, an active reconstruction agent that leverages feed-forward 3D reconstruction models and vision-language guidance.<n>Our framework decouples view-uncertainty modeling from the underlying feed-forward reconstructor, enabling precise uncertainty estimation without expensive online optimization.
  arXiv  Detail & Related papers  (2025-11-28T06:17:02Z)
• Monocular 3D Object Reconstruction with GAN Inversion [122.96094885939146]
  MeshInversion is a novel framework to improve the reconstruction of textured 3D meshes. It exploits the generative prior of a 3D GAN pre-trained for 3D textured mesh synthesis. Our framework obtains faithful 3D reconstructions with consistent geometry and texture across both observed and unobserved parts.
  arXiv  Detail & Related papers  (2022-07-20T17:47:22Z)
• Reconstruct, Rasterize and Backprop: Dense shape and pose estimation from a single image [14.9851111159799]
  This paper presents a new system to obtain dense object reconstructions along with 6-DoF poses from a single image. We leverage recent advances in differentiable rendering (in particular, robotics) to close the loop with 3D reconstruction in camera frame.
  arXiv  Detail & Related papers  (2020-04-25T20:53:43Z)
• Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image [102.44347847154867]
  We propose a novel formulation that allows to jointly recover the geometry of a 3D object as a set of primitives. Our model recovers the higher level structural decomposition of various objects in the form of a binary tree of primitives. Our experiments on the ShapeNet and D-FAUST datasets demonstrate that considering the organization of parts indeed facilitates reasoning about 3D geometry.
  arXiv  Detail & Related papers  (2020-04-02T17:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.