Trans2Occ: Voxel Occupancy Estimation and Grasp for Transparent Objects from Simulation to Reality

• URL: http://arxiv.org/abs/2606.01777v1
• Date: Mon, 01 Jun 2026 06:59:13 GMT
• Title: Trans2Occ: Voxel Occupancy Estimation and Grasp for Transparent Objects from Simulation to Reality
• Abstract summary: We present a framework for transparent object perception and manipulation based on single-view RGB input.<n>Our approach predicts voxel-space occupancy directly from a single image, providing a geometry-aware representation that supports downstream robotic grasping.
• Score: 125.76063573012853
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transparent objects remain challenging for robotic perception due to unreliable depth sensing caused by refraction and reflection. While prior approaches rely on multi-view reconstruction or depth completion, they are often difficult to scale or deploy in real-world robotic systems. In this paper, we present a practical framework for transparent object perception and manipulation based on single-view RGB input. Our approach predicts voxel-space occupancy directly from a single image, providing a geometry-aware representation that supports downstream robotic grasping. To enable large-scale training, we construct a simulation pipeline that generates paired RGB images and voxel occupancy annotations under diverse materials and lighting conditions. We demonstrate that the predicted occupancy representation is robust to domain shifts and transfers effectively from simulation to real-world robotic setups without fine-tuning. A simple rule-based grasping strategy built on top of the occupancy further achieves reliable grasp performance on transparent objects. Extensive experiments in both simulation and real-world environments show that our framework provides accurate 3D understanding and enables practical manipulation of transparent objects. These results suggest that single-view occupancy prediction offers a scalable and effective solution for transparent object perception in robotics.

Related papers

• Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator [70.34667754804168]
  Embodied navigation aims to build agents that interpret multimodal goals, reason in 3D space, and reach target destinations reliably in the real world.<n>We introduce Image2Sim, a real-time neural simulation framework that constructs high-quality interactive environments from posed RGB-D image sequences.
  arXiv  Detail & Related papers  (2026-07-07T02:42:41Z)
• R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation [74.41728218960465]
  We propose a real-to-real 3D data generation framework (R2RGen) that directly augments the pointcloud observation-action pairs to generate real-world data.<n>R2RGen substantially enhances data efficiency on extensive experiments and demonstrates strong potential for scaling and application on mobile manipulation.
  arXiv  Detail & Related papers  (2025-10-09T17:55:44Z)
• ManipGPT: Is Affordance Segmentation by Large Vision Models Enough for Articulated Object Manipulation? [23.73409728939164]
  This paper introduces ManipGPT, a framework designed to predict optimal interaction areas for articulated objects.<n>We create a dataset of 9.9k simulated and real images to bridge the visual sim-to-real gap.<n>We significantly improve part-level affordance segmentation, adapting the model's in-context segmentation capabilities to robot manipulation scenarios.
  arXiv  Detail & Related papers  (2024-12-13T11:22:01Z)
• ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation [18.140839442955485]
  We develop a vision transformer-based algorithm for stereo depth recovery of transparent objects.<n>Our method incorporates a parameter-aligned, domain-adaptive, and physically realistic Sim2Real simulation for efficient data generation.<n>Our experimental results demonstrate the model's exceptional Sim2Real generalizability in real-world scenarios.
  arXiv  Detail & Related papers  (2024-09-13T15:44:38Z)
• RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation [50.10282876199739]
  This paper introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects. It integrates the RFNet, which predicts refractive flow, object mask, and boundaries, followed by the F2Net, which estimates surface normal from the refractive flow. A real-world robot grasping task witnesses an 83% success rate, proving that refractive flow can help enable direct sim-to-real transfer.
  arXiv  Detail & Related papers  (2023-11-21T07:19:47Z)
• Learning Sim-to-Real Dense Object Descriptors for Robotic Manipulation [4.7246285569677315]
  We present Sim-to-Real Dense Object Nets (SRDONs), a dense object descriptor that not only understands the object via appropriate representation but also maps simulated and real data to a unified feature space with pixel consistency. We demonstrate in experiments that pre-trained SRDONs significantly improve performances on unseen objects and unseen visual environments for various robotic tasks with zero real-world training.
  arXiv  Detail & Related papers  (2023-04-18T02:28:55Z)
• SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo [4.317104502755003]
  SimNet is trained as a single multi-headed neural network using simulated stereo data. SimNet is evaluated on 2D car detection, unknown object detection, and deformable object keypoint detection. By inferring grasp positions using the OBB and keypoint predictions, SimNet can be used to perform end-to-end manipulation of unknown objects.
  arXiv  Detail & Related papers  (2021-06-30T15:18:14Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)
• Transferable Active Grasping and Real Embodied Dataset [48.887567134129306]
  We show how to search for feasible viewpoints for grasping by the use of hand-mounted RGB-D cameras. A practical 3-stage transferable active grasping pipeline is developed, that is adaptive to unseen clutter scenes. In our pipeline, we propose a novel mask-guided reward to overcome the sparse reward issue in grasping and ensure category-irrelevant behavior.
  arXiv  Detail & Related papers  (2020-04-28T08:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.