Guiding Image-to-3D Generation with Test-Time Partial Observations

• URL: http://arxiv.org/abs/2609.10531v1
• Date: Wed, 09 Sep 2026 17:58:04 GMT
• Title: Guiding Image-to-3D Generation with Test-Time Partial Observations
• Abstract summary: We introduce a training-free framework for incorporating partial geometric observations into pretrained image-to-3D generative models without retraining or finetuning.<n>Our results demonstrate that pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance.
• Score: 10.438161423840938
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Image-to-3D models can generate visually compelling 3D assets from a single RGB image, but their geometry is often only loosely constrained by the available observations, limiting their use in applications that require geometric fidelity. In many real-world settings, however, partial geometric observations of the object may be available at test time. We introduce a training-free framework for incorporating such evidence into pretrained image-to-3D generative models without retraining or finetuning. To do this, we guide generation using a ray-consistent observation likelihood defined over the model's occupancy representation, combining surface occupancy and free-space evidence. Applied to SAM 3D and its multi-view extension, our approach substantially improves geometric fidelity across different levels of observability, as well as visual quality. Our results demonstrate that pretrained image-to-3D models can effectively integrate partial geometric observations through explicit test-time guidance, complementing their learned generative priors without modifying the underlying model.

Related papers

• Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors [61.34273238077091]
  We present a novel method for generating orbital videos from a single image of an object.<n>Our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-14T05:35:46Z)
• E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training [55.61379509038588]
  We present E-RayZer, a self-supervised large 3D Vision model that learns truly 3D-aware representations directly from unlabeled images.<n>E-RayZer operates directly in 3D space, performing self-supervised 3D reconstruction with Explicit geometry.
  arXiv  Detail & Related papers  (2025-12-11T18:59:53Z)
• Geometry and Perception Guided Gaussians for Multiview-consistent 3D Generation from a Single Image [10.648593818811976]
  Existing approaches often rely on fine-tuning pretrained 2D diffusion models or directly generating 3D information through fast network inference.<n>We present a novel method that seamlessly integrates geometry and perception information without requiring additional model training.<n> Experimental results show that we outperform existing methods on novel view synthesis and 3D reconstruction, demonstrating robust and consistent 3D object generation.
  arXiv  Detail & Related papers  (2025-06-26T11:22:06Z)
• Shape from Semantics: 3D Shape Generation from Multi-View Semantics [30.969299308083723]
  Existing 3D reconstruction methods utilize guidances such as 2D images, 3D point clouds, shape contours and single semantics to recover the 3D surface.<n>We propose a novel 3D modeling task called Shape from Semantics'', which aims to create 3D models whose geometry and appearance are consistent with the given text semantics when viewed from different views.
  arXiv  Detail & Related papers  (2025-02-01T07:51:59Z)
• CtrlNeRF: The Generative Neural Radiation Fields for the Controllable Synthesis of High-fidelity 3D-Aware Images [5.50550810374347]
  generative neural radiance field (GRAF) is capable of producing images from random noise z without 3D supervision.<n>In practice, shape and appearance are modeled by z_s and z_a, respectively, to manipulate them separately during inference.<n>We introduce a controllable generative model that uses a single network to represent multiple scenes with shared weights.
  arXiv  Detail & Related papers  (2024-12-01T10:19:24Z)
• DreamPolish: Domain Score Distillation With Progressive Geometry Generation [66.94803919328815]
  We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures. In the geometry construction phase, our approach leverages multiple neural representations to enhance the stability of the synthesis process. In the texture generation phase, we introduce a novel score distillation objective, namely domain score distillation (DSD), to guide neural representations toward such a domain.
  arXiv  Detail & Related papers  (2024-11-03T15:15:01Z)
• G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images [45.66479596827045]
  We propose a Geometry-enhanced NeRF (G-NeRF), which seeks to enhance the geometry priors by a geometry-guided multi-view synthesis approach. To tackle the absence of multi-view supervision for single-view images, we design the depth-aware training approach.
  arXiv  Detail & Related papers  (2024-04-11T04:58:18Z)
• Structured 3D Features for Reconstructing Controllable Avatars [43.36074729431982]
  We introduce Structured 3D Features, a model based on a novel implicit 3D representation that pools pixel-aligned image features onto dense 3D points sampled from a parametric, statistical human mesh surface. We show that our S3F model surpasses the previous state-of-the-art on various tasks, including monocular 3D reconstruction, as well as albedo and shading estimation.
  arXiv  Detail & Related papers  (2022-12-13T18:57:33Z)
• Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images [82.32776379815712]
  We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. We adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable function for test-time optimization.
  arXiv  Detail & Related papers  (2022-04-21T03:42:31Z)
• Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Images [94.49117671450531]
  State-of-the-art 3D generative models are GANs which use neural 3D volumetric representations for synthesis. In this paper, we design a 3D GAN which can learn a disentangled model of objects, just from monocular observations.
  arXiv  Detail & Related papers  (2022-03-29T22:03:18Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.