SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation

• URL: http://arxiv.org/abs/2602.23359v1
• Date: Thu, 26 Feb 2026 18:59:05 GMT
• Title: SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation
• Authors: Vaibhav Agrawal, Rishubh Parihar, Pradhaan Bhat, Ravi Kiran Sarvadevabhatla, R. Venkatesh Babu,
• Abstract summary: occlusion reasoning is essential for partially occluded objects with depth-consistent geometry and scale.<n>We propose SeeThrough3D, a model for 3D layout conditioned generation that explicitly models occlusions.
• Score: 32.15143378003745
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We identify occlusion reasoning as a fundamental yet overlooked aspect for 3D layout-conditioned generation. It is essential for synthesizing partially occluded objects with depth-consistent geometry and scale. While existing methods can generate realistic scenes that follow input layouts, they often fail to model precise inter-object occlusions. We propose SeeThrough3D, a model for 3D layout conditioned generation that explicitly models occlusions. We introduce an occlusion-aware 3D scene representation (OSCR), where objects are depicted as translucent 3D boxes placed within a virtual environment and rendered from desired camera viewpoint. The transparency encodes hidden object regions, enabling the model to reason about occlusions, while the rendered viewpoint provides explicit camera control during generation. We condition a pretrained flow based text-to-image image generation model by introducing a set of visual tokens derived from our rendered 3D representation. Furthermore, we apply masked self-attention to accurately bind each object bounding box to its corresponding textual description, enabling accurate generation of multiple objects without object attribute mixing. To train the model, we construct a synthetic dataset with diverse multi-object scenes with strong inter-object occlusions. SeeThrough3D generalizes effectively to unseen object categories and enables precise 3D layout control with realistic occlusions and consistent camera control.

Related papers

• RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roaming [79.81527946524098]
  RoamScene3D is a novel framework that bridges the gap between semantic guidance and spatial generation.<n>We employ a vision-language model (VLM) to construct a scene graph that encodes object relations.<n>To mitigate the limitations of static 2D priors, we introduce a Motion-Injected Inpainting model that is fine-tuned on a synthetic panoramic dataset.
  arXiv  Detail & Related papers  (2026-01-27T10:10:55Z)
• SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model [83.37403036061403]
  We propose a decoupled 3D scene generation framework called SceneMaker in this work.<n>We first decouple the de-occlusion model from 3D object generation, and enhance it by leveraging image datasets and collected de-occlusion datasets.<n>Then, we propose a unified pose estimation model that integrates global and local mechanisms for both self-attention and cross-attention to improve accuracy.
  arXiv  Detail & Related papers  (2025-12-11T18:59:56Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• Disentangled 3D Scene Generation with Layout Learning [109.03233745767062]
  We introduce a method to generate 3D scenes that are disentangled into their component objects. Our key insight is that objects can be discovered by finding parts of a 3D scene that, when rearranged spatially, still produce valid configurations of the same scene. We show that despite its simplicity, our approach successfully generates 3D scenes into individual objects.
  arXiv  Detail & Related papers  (2024-02-26T18:54:15Z)
• SceneWiz3D: Towards Text-guided 3D Scene Composition [134.71933134180782]
  Existing approaches either leverage large text-to-image models to optimize a 3D representation or train 3D generators on object-centric datasets. We introduce SceneWiz3D, a novel approach to synthesize high-fidelity 3D scenes from text.
  arXiv  Detail & Related papers  (2023-12-13T18:59:30Z)
• DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis [90.32352050266104]
  DisCoScene is a 3Daware generative model for high-quality and controllable scene synthesis. It disentangles the whole scene into object-centric generative fields by learning on only 2D images with the global-local discrimination. We demonstrate state-of-the-art performance on many scene datasets, including the challenging outdoor dataset.
  arXiv  Detail & Related papers  (2022-12-22T18:59:59Z)
• Volumetric Disentanglement for 3D Scene Manipulation [22.22326242219791]
  We propose a volumetric framework for disentangling or separating, the volumetric representation of a given foreground object from the background, and semantically manipulating the foreground object, as well as the background. Our framework takes as input a set of 2D masks specifying the desired foreground object for training views, together with the associated 2D views and poses, and produces a foreground-background disentanglement. We subsequently demonstrate the applicability of our framework on a number of downstream manipulation tasks including object camouflage, non-negative 3D object inpainting, 3D object translation, 3D object inpainting, and 3D text-based
  arXiv  Detail & Related papers  (2022-06-06T17:57:07Z)
• CoCoNets: Continuous Contrastive 3D Scene Representations [21.906643302668716]
  This paper explores self-supervised learning of amodal 3D feature representations from RGB and RGB-D posed images and videos. We show the resulting 3D visual feature representations effectively scale across objects and scenes, imagine information occluded or missing from the input viewpoints, track objects over time, align semantically related objects in 3D, and improve 3D object detection.
  arXiv  Detail & Related papers  (2021-04-08T15:50:47Z)
• ROOTS: Object-Centric Representation and Rendering of 3D Scenes [28.24758046060324]
  A crucial ability of human intelligence is to build up models of individual 3D objects from partial scene observations. Recent works achieve object-centric generation but without the ability to infer the representation. We propose a probabilistic generative model for learning to build modular and compositional 3D object models.
  arXiv  Detail & Related papers  (2020-06-11T00:42:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.