S-INF: Towards Realistic Indoor Scene Synthesis via Scene Implicit Neural Field

• URL: http://arxiv.org/abs/2412.17561v2
• Date: Sat, 04 Jan 2025 08:42:33 GMT
• Title: S-INF: Towards Realistic Indoor Scene Synthesis via Scene Implicit Neural Field
• Authors: Zixi Liang, Guowei Xu, Haifeng Wu, Ye Huang, Wen Li, Lixin Duan,
• Abstract summary: We introduce Scene Implicit Neural Field (S-INF) for indoor scene synthesis, aiming to learn meaningful representations of multimodal relationships. S-INF disentangles the multimodal relationships into scene layout relationships and detailed object relationships, fusing them later through implicit neural fields. It consistently achieves state-of-the-art performance under different types of indoor scene synthesis.
• Score: 25.539007827647737
• License:
• Abstract: Learning-based methods have become increasingly popular in 3D indoor scene synthesis (ISS), showing superior performance over traditional optimization-based approaches. These learning-based methods typically model distributions on simple yet explicit scene representations using generative models. However, due to the oversimplified explicit representations that overlook detailed information and the lack of guidance from multimodal relationships within the scene, most learning-based methods struggle to generate indoor scenes with realistic object arrangements and styles. In this paper, we introduce a new method, Scene Implicit Neural Field (S-INF), for indoor scene synthesis, aiming to learn meaningful representations of multimodal relationships, to enhance the realism of indoor scene synthesis. S-INF assumes that the scene layout is often related to the object-detailed information. It disentangles the multimodal relationships into scene layout relationships and detailed object relationships, fusing them later through implicit neural fields (INFs). By learning specialized scene layout relationships and projecting them into S-INF, we achieve a realistic generation of scene layout. Additionally, S-INF captures dense and detailed object relationships through differentiable rendering, ensuring stylistic consistency across objects. Through extensive experiments on the benchmark 3D-FRONT dataset, we demonstrate that our method consistently achieves state-of-the-art performance under different types of ISS.

Related papers

• Hierarchically-Structured Open-Vocabulary Indoor Scene Synthesis with Pre-trained Large Language Model [14.70850176122733]
  We propose to generate hierarchically structured scene descriptions with large language models (LLM) and then compute the scene layouts. Specifically, we train a hierarchy-aware network to infer the fine-grained relative positions between objects. We also present open-vocabulary scene synthesis and interactive scene design results to show the strength of our approach in the applications.
  arXiv  Detail & Related papers  (2025-02-15T05:04:14Z)
• InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior [27.773451301040424]
  InstructScene is a novel generative framework that integrates a semantic graph prior and a layout decoder. We show that the proposed method surpasses existing state-of-the-art approaches by a large margin.
  arXiv  Detail & Related papers  (2024-02-07T10:09:00Z)
• CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion [83.30168660888913]
  We present CommonScenes, a fully generative model that converts scene graphs into corresponding controllable 3D scenes. Our pipeline consists of two branches, one predicting the overall scene layout via a variational auto-encoder and the other generating compatible shapes. The generated scenes can be manipulated by editing the input scene graph and sampling the noise in the diffusion model.
  arXiv  Detail & Related papers  (2023-05-25T17:39:13Z)
• LANe: Lighting-Aware Neural Fields for Compositional Scene Synthesis [65.20672798704128]
  We present Lighting-Aware Neural Field (LANe) for compositional synthesis of driving scenes. We learn a scene representation that disentangles the static background and transient elements into a world-NeRF and class-specific object-NeRFs. We demonstrate the performance of our model on a synthetic dataset of diverse lighting conditions rendered with the CARLA simulator.
  arXiv  Detail & Related papers  (2023-04-06T17:59:25Z)
• DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis [90.32352050266104]
  DisCoScene is a 3Daware generative model for high-quality and controllable scene synthesis. It disentangles the whole scene into object-centric generative fields by learning on only 2D images with the global-local discrimination. We demonstrate state-of-the-art performance on many scene datasets, including the challenging outdoor dataset.
  arXiv  Detail & Related papers  (2022-12-22T18:59:59Z)
• Towards 3D Scene Understanding by Referring Synthetic Models [65.74211112607315]
  Methods typically alleviate on-extensive annotations on real scene scans. We explore how synthetic models rely on real scene categories of synthetic features to a unified feature space. Experiments show that our method achieves the average mAP of 46.08% on the ScanNet S3DIS dataset and 55.49% by learning datasets.
  arXiv  Detail & Related papers  (2022-03-20T13:06:15Z)
• DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization [66.25948693095604]
  We propose a novel method for panoramic 3D scene understanding which recovers the 3D room layout and the shape, pose, position, and semantic category for each object from a single full-view panorama image. Experiments demonstrate that our method outperforms existing methods on panoramic scene understanding in terms of both geometry accuracy and object arrangement.
  arXiv  Detail & Related papers  (2021-08-24T13:55:29Z)
• Neural Scene Graphs for Dynamic Scenes [57.65413768984925]
  We present the first neural rendering method that decomposes dynamic scenes into scene graphs. We learn implicitly encoded scenes, combined with a jointly learned latent representation to describe objects with a single implicit function.
  arXiv  Detail & Related papers  (2020-11-20T12:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.