Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier
Transform
- URL: http://arxiv.org/abs/2402.18287v1
- Date: Wed, 28 Feb 2024 12:27:28 GMT
- Title: Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier
Transform
- Authors: Bruno Henriques, Benjamin Allaert, Jean-Philippe Vandeborre
- Abstract summary: Inpainting indoor environments from a single image plays a crucial role in modeling the internal structure of interior spaces.
We propose an innovative approach based on a U-Former architecture and a new Windowed-FourierMixer block.
This new architecture proves advantageous for tasks involving indoor scenes where symmetry is prevalent.
- Score: 3.864321514889099
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: With the growing demand for immersive digital applications, the need to
understand and reconstruct 3D scenes has significantly increased. In this
context, inpainting indoor environments from a single image plays a crucial
role in modeling the internal structure of interior spaces as it enables the
creation of textured and clutter-free reconstructions. While recent methods
have shown significant progress in room modeling, they rely on constraining
layout estimators to guide the reconstruction process. These methods are highly
dependent on the performance of the structure estimator and its generative
ability in heavily occluded environments. In response to these issues, we
propose an innovative approach based on a U-Former architecture and a new
Windowed-FourierMixer block, resulting in a unified, single-phase network
capable of effectively handle human-made periodic structures such as indoor
spaces. This new architecture proves advantageous for tasks involving indoor
scenes where symmetry is prevalent, allowing the model to effectively capture
features such as horizon/ceiling height lines and cuboid-shaped rooms.
Experiments show the proposed approach outperforms current state-of-the-art
methods on the Structured3D dataset demonstrating superior performance in both
quantitative metrics and qualitative results. Code and models will be made
publicly available.
Related papers
- Mixed Diffusion for 3D Indoor Scene Synthesis [55.94569112629208]
We present MiDiffusion, a novel mixed discrete-continuous diffusion model architecture.
We represent a scene layout by a 2D floor plan and a set of objects, each defined by its category, location, size, and orientation.
Our experimental results demonstrate that MiDiffusion substantially outperforms state-of-the-art autoregressive and diffusion models in floor-conditioned 3D scene synthesis.
arXiv Detail & Related papers (2024-05-31T17:54:52Z) - VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for
Enhanced Indoor View Synthesis [51.49008959209671]
We introduce VoxNeRF, a novel approach that leverages volumetric representations to enhance the quality and efficiency of indoor view synthesis.
We employ multi-resolution hash grids to adaptively capture spatial features, effectively managing occlusions and the intricate geometry of indoor scenes.
We validate our approach against three public indoor datasets and demonstrate that VoxNeRF outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-11-09T11:32:49Z) - Human as Points: Explicit Point-based 3D Human Reconstruction from
Single-view RGB Images [78.56114271538061]
We introduce an explicit point-based human reconstruction framework called HaP.
Our approach is featured by fully-explicit point cloud estimation, manipulation, generation, and refinement in the 3D geometric space.
Our results may indicate a paradigm rollback to the fully-explicit and geometry-centric algorithm design.
arXiv Detail & Related papers (2023-11-06T05:52:29Z) - Exploiting Multiple Priors for Neural 3D Indoor Reconstruction [15.282699095607594]
We propose a novel neural implicit modeling method that leverages multiple regularization strategies to achieve better reconstructions of large indoor environments.
Experimental results show that our approach produces state-of-the-art 3D reconstructions in challenging indoor scenarios.
arXiv Detail & Related papers (2023-09-13T15:23:43Z) - Neural 3D Reconstruction in the Wild [86.6264706256377]
We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections.
We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
arXiv Detail & Related papers (2022-05-25T17:59:53Z) - Reconstructing Compact Building Models from Point Clouds Using Deep
Implicit Fields [4.683612295430956]
We present a novel framework for reconstructing compact, watertight, polygonal building models from point clouds.
Experiments on both synthetic and real-world point clouds have demonstrated that, with our neural-guided strategy, high-quality building models can be obtained with significant advantages in fidelity, compactness, and computational efficiency.
arXiv Detail & Related papers (2021-12-24T21:32:32Z) - PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes [0.0]
Diminished Reality (DR) fulfills the requirement of such applications, to remove existing objects in the scene.
To preserve the reality' in indoor (re-)planning applications, the scene's structure preservation is crucial.
We propose a model that initially predicts the structure of an indoor scene and then uses it to guide the reconstruction of an empty -- background only -- representation of the same scene.
arXiv Detail & Related papers (2021-06-01T12:56:53Z) - Enhanced 3D Human Pose Estimation from Videos by using Attention-Based
Neural Network with Dilated Convolutions [12.900524511984798]
We show a systematic design for how conventional networks and other forms of constraints can be incorporated into the attention framework.
We achieve this by adapting temporal receptive field via a multi-scale structure of dilated convolutions.
Our method achieves the state-of-the-art performance and outperforms existing methods by reducing the mean per joint position error to 33.4 mm on Human3.6M dataset.
arXiv Detail & Related papers (2021-03-04T17:26:51Z) - S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures.
We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
arXiv Detail & Related papers (2020-07-13T17:44:30Z) - Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure
Reconstruction from an RGB Video [90.93141123721713]
Thin structures, such as wire-frame sculptures, fences, cables, power lines, and tree branches, are common in the real world.
It is extremely challenging to acquire their 3D digital models using traditional image-based or depth-based reconstruction methods because thin structures often lack distinct point features and have severe self-occlusion.
We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera.
arXiv Detail & Related papers (2020-05-07T10:39:20Z) - Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors [30.262308825799167]
We show that complex encoder-decoder architectures perform similarly to nearest-neighbor baselines in standard benchmarks.
We propose three approaches that efficiently integrate a class prior into a 3D reconstruction model.
arXiv Detail & Related papers (2020-04-14T04:53:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.