Multi-Agent Reinforcement Learning of 3D Furniture Layout Simulation in Indoor Graphics Scenes

• URL: http://arxiv.org/abs/2102.09137v1
• Date: Thu, 18 Feb 2021 03:20:35 GMT
• Title: Multi-Agent Reinforcement Learning of 3D Furniture Layout Simulation in Indoor Graphics Scenes
• Authors: Xinhan Di, Pengqian Yu
• Abstract summary: We explore the interior graphics scenes design task as a Markov decision process (MDP) in 3D simulation. The goal is to produce furniture layout in the 3D simulation of the indoor graphics scenes. We conduct experiments on a large-scale real-world interior layout dataset.
• Score: 3.4447129363520332
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the industrial interior design process, professional designers plan the furniture layout to achieve a satisfactory 3D design for selling. In this paper, we explore the interior graphics scenes design task as a Markov decision process (MDP) in 3D simulation, which is solved by multi-agent reinforcement learning. The goal is to produce furniture layout in the 3D simulation of the indoor graphics scenes. In particular, we firstly transform the 3D interior graphic scenes into two 2D simulated scenes. We then design the simulated environment and apply two reinforcement learning agents to learn the optimal 3D layout for the MDP formulation in a cooperative way. We conduct our experiments on a large-scale real-world interior layout dataset that contains industrial designs from professional designers. Our numerical results demonstrate that the proposed model yields higher-quality layouts as compared with the state-of-art model. The developed simulator and codes are available at \url{https://github.com/CODE-SUBMIT/simulator2}.

Related papers

• Mixed Diffusion for 3D Indoor Scene Synthesis [55.94569112629208]
  We present MiDiffusion, a novel mixed discrete-continuous diffusion model architecture. We represent a scene layout by a 2D floor plan and a set of objects, each defined by its category, location, size, and orientation. Our experimental results demonstrate that MiDiffusion substantially outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis.
  arXiv  Detail & Related papers  (2024-05-31T17:54:52Z)
• ControlRoom3D: Room Generation using Semantic Proxy Rooms [48.93419701713694]
  We present ControlRoom3D, a novel method to generate high-quality room meshes. Our approach is a user-defined 3D semantic proxy room that outlines a rough room layout. When rendered to 2D, this 3D representation provides valuable geometric and semantic information to control powerful 2D models.
  arXiv  Detail & Related papers  (2023-12-08T17:55:44Z)
• PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm [114.47216525866435]
  We introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation. For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness.
  arXiv  Detail & Related papers  (2023-10-12T17:59:57Z)
• Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training [105.3421541518582]
  Current successful methods of 3D scene perception rely on the large-scale annotated point cloud. We propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages. Model2Scene yields impressive label-free 3D object salient detection with an average mAP of 46.08% and 55.49% on the ScanNet and S3DIS datasets, respectively.
  arXiv  Detail & Related papers  (2023-09-29T03:51:26Z)
• CC3D: Layout-Conditioned Generation of Compositional 3D Scenes [49.281006972028194]
  We introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts. Our evaluations on synthetic 3D-FRONT and real-world KITTI-360 datasets demonstrate that our model generates scenes of improved visual and geometric quality.
  arXiv  Detail & Related papers  (2023-03-21T17:59:02Z)
• Hierarchical Reinforcement Learning for Furniture Layout in Virtual Indoor Scenes [2.2481284426718533]
  In this paper, we explore the furniture layout task as a Markov decision process (MDP) in virtual reality. The goal is to produce a proper two-furniture layout in the virtual reality of the indoor scenes.
  arXiv  Detail & Related papers  (2022-10-19T09:58:10Z)
• Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation [10.024367148266721]
  This paper focuses on volumetric design generation conditioned on an input program graph. Instead of outputting dense 3D voxels, we propose a new 3D representation named voxel graph that is both compact and expressive for building geometries. Our generator is a cross-modal graph neural network that uses a pointer mechanism to connect the input program graph and the output voxel graph, and the whole pipeline is trained using the adversarial framework.
  arXiv  Detail & Related papers  (2021-04-27T16:49:34Z)
• 3D-to-2D Distillation for Indoor Scene Parsing [78.36781565047656]
  We present a new approach that enables us to leverage 3D features extracted from large-scale 3D data repository to enhance 2D features extracted from RGB images. First, we distill 3D knowledge from a pretrained 3D network to supervise a 2D network to learn simulated 3D features from 2D features during the training. Second, we design a two-stage dimension normalization scheme to calibrate the 2D and 3D features for better integration. Third, we design a semantic-aware adversarial training model to extend our framework for training with unpaired 3D data.
  arXiv  Detail & Related papers  (2021-04-06T02:22:24Z)
• Deep Reinforcement Learning for Producing Furniture Layout in Indoor Scenes [3.4447129363520332]
  In the industrial interior design process, professional designers plan the size and position of furniture in a room to achieve a satisfactory design for selling. We explore the interior scene design task as a Markov decision process (MDP), which is solved by deep reinforcement learning. Our numerical results demonstrate that the proposed model yields higher-quality layouts as compared with the state-of-art model.
  arXiv  Detail & Related papers  (2021-01-19T04:38:58Z)
• 3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics [21.660708913700184]
  3D-FRONT contains 18,968 rooms diversely furnished by 3D objects, far surpassing all publicly available scene datasets. We release Trescope, a light-weight rendering tool, to support benchmark rendering of 2D images and annotations from 3D-FRONT.
  arXiv  Detail & Related papers  (2020-11-18T07:14:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.