Seeing Before Generating: Object Perception Enhances Single-View 3D Reconstruction

• URL: http://arxiv.org/abs/2607.18630v1
• Date: Tue, 21 Jul 2026 01:55:24 GMT
• Title: Seeing Before Generating: Object Perception Enhances Single-View 3D Reconstruction
• Abstract summary: We show that learnt object perception can significantly enhance 3D reconstruction.<n>We propose a method that leverages perceptual signals extracted from pretrained perception models.<n>Our approach is model-agnostic and can be integrated into various reconstruction methods in a plug-and-play manner.
• Score: 6.264701896062822
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The relationship between object perception and reconstruction is well established in human vision, yet remains underexplored in computer vision. In this paper, we demonstrate that learnt object perception can significantly enhance 3D reconstruction. Focusing on the challenging task of single-view 3D object reconstruction, we propose a method that leverages perceptual signals extracted from pretrained perception models capturing semantic and geometric information to drive the reconstruction of an object from its single image. Our approach is model-agnostic and can be integrated into various reconstruction methods in a plug-and-play manner. Experiments with two state-of-the-art single-view 3D reconstruction pipelines in a benchmark dataset show consistent and substantial improvements achieved by our method, validating the effectiveness of incorporating perception into generation. We provide in-depth analysis of various aspects of our method and its application. Our project page is at https://ynhuhuynh.github.io/perception-3d/.

Related papers

• When Does An Extra View Help? Adapting Single-View 3D Reconstruction with Extra Imagery [9.55951636411851]
  ASV3D is a framework for adapting single-view 3D object reconstruction to test-time data with support from one additional image.<n>We apply our ASV3D to improve two state-of-the-art single-view 3D reconstruction pipelines on both benchmark and real-world datasets.
  arXiv  Detail & Related papers  (2026-08-08T13:39:28Z)
• Pano3D: Unified 3D Reconstruction and Panoptic Segmentation [68.18993232814061]
  We introduce an approach that performs 3D reconstruction and 3D panoptic segmentation in a unified framework.<n>We build on existing 3D reconstruction models and augment them with a set-based mask decoder.<n>Our method achieves state-of-the-art performance in 3D panoptic segmentation across ScanNet, ScanNet200, and ScanNet++ datasets.
  arXiv  Detail & Related papers  (2026-06-12T09:43:13Z)
• DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion [65.65756111062005]
  textitDecoRec is a system designed to elevate single-view 2D images to a decomposed 3D scene mesh.<n>Our results demonstrate that DecoRec facilitates high-quality single-view scene reconstruction in both geometry and novel synthesis.
  arXiv  Detail & Related papers  (2026-05-16T04:23:48Z)
• ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Map [11.874184782686532]
  We propose the first approach for realistic 3D hand-object shape and pose reconstruction from a single depth map. Our pipeline additionally predicts voxelized hand-object shapes, having a one-to-one mapping to the input voxelized depth. In addition, we show the impact of adding another GraFormer component that refines the reconstructed shapes based on the hand-object interactions.
  arXiv  Detail & Related papers  (2023-10-18T09:05:57Z)
• 3D-LatentMapper: View Agnostic Single-View Reconstruction of 3D Shapes [0.0]
  We propose a novel framework that leverages the intermediate latent spaces of Vision Transformer (ViT) and a joint image-text representational model, CLIP, for fast and efficient Single View Reconstruction (SVR) We use the ShapeNetV2 dataset and perform extensive experiments with comparisons to SOTA methods to demonstrate our method's effectiveness.
  arXiv  Detail & Related papers  (2022-12-05T11:45:26Z)
• Reconstructing Hand-Held Objects from Monocular Video [95.06750686508315]
  This paper presents an approach that reconstructs a hand-held object from a monocular video. In contrast to many recent methods that directly predict object geometry by a trained network, the proposed approach does not require any learned prior to the object.
  arXiv  Detail & Related papers  (2022-11-30T09:14:58Z)
• State of the Art in Dense Monocular Non-Rigid 3D Reconstruction [100.9586977875698]
  3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views.
  arXiv  Detail & Related papers  (2022-10-27T17:59:53Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• Object Wake-up: 3-D Object Reconstruction, Animation, and in-situ Rendering from a Single Image [58.69732754597448]
  Given a picture of a chair, could we extract the 3-D shape of the chair, animate its plausible articulations and motions, and render in-situ in its original image space? We devise an automated approach to extract and manipulate articulated objects in single images.
  arXiv  Detail & Related papers  (2021-08-05T16:20:12Z)
• A Divide et Impera Approach for 3D Shape Reconstruction from Multiple Views [49.03830902235915]
  Estimating the 3D shape of an object from a single or multiple images has gained popularity thanks to the recent breakthroughs powered by deep learning. This paper proposes to rely on viewpoint variant reconstructions by merging the visible information from the given views. To validate the proposed method, we perform a comprehensive evaluation on the ShapeNet reference benchmark in terms of relative pose estimation and 3D shape reconstruction.
  arXiv  Detail & Related papers  (2020-11-17T09:59:32Z)
• 3D Reconstruction of Novel Object Shapes from Single Images [23.016517962380323]
  We show that our proposed SDFNet achieves state-of-the-art performance on seen and unseen shapes. We provide the first large-scale evaluation of single image shape reconstruction to unseen objects.
  arXiv  Detail & Related papers  (2020-06-14T00:34:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.