Sound field reconstruction in rooms: inpainting meets super-resolution

• URL: http://arxiv.org/abs/2001.11263v2
• Date: Thu, 6 Aug 2020 16:14:24 GMT
• Title: Sound field reconstruction in rooms: inpainting meets super-resolution
• Authors: Francesc Llu\'is, Pablo Mart\'inez-Nuevo, Martin Bo M{\o}ller, Sven Ewan Shepstone
• Abstract summary: Deep-learning method for sound field reconstruction is proposed. The method is based on a U-net-like neural network with partial convolutions trained solely on simulated data. Experiments using simulated data together with an experimental validation in a real listening room are shown.
• Score: 1.0705399532413618
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper, a deep-learning-based method for sound field reconstruction is proposed. It is shown the possibility to reconstruct the magnitude of the sound pressure in the frequency band 30-300 Hz for an entire room by using a very low number of irregularly distributed microphones arbitrarily arranged. Moreover, the approach is agnostic to the location of the measurements in the Euclidean space. In particular, the presented approach uses a limited number of arbitrary discrete measurements of the magnitude of the sound field pressure in order to extrapolate this field to a higher-resolution grid of discrete points in space with a low computational complexity. The method is based on a U-net-like neural network with partial convolutions trained solely on simulated data, which itself is constructed from numerical simulations of Green's function across thousands of common rectangular rooms. Although extensible to three dimensions and different room shapes, the method focuses on reconstructing a two-dimensional plane of a rectangular room from measurements of the three-dimensional sound field. Experiments using simulated data together with an experimental validation in a real listening room are shown. The results suggest a performance which may exceed conventional reconstruction techniques for a low number of microphones and computational requirements.

Related papers

• N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
  In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures. We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our method provides faithful approximations of visibility, depth, and appearance attributes.
  arXiv  Detail & Related papers  (2024-05-25T13:54:34Z)
• Room Transfer Function Reconstruction Using Complex-valued Neural Networks and Irregularly Distributed Microphones [15.396703290586418]
  We employ complex-valued neural networks to estimate room transfer functions in the frequency range of the first room resonances. This is the first time that complex-valued neural networks are used to estimate room transfer functions.
  arXiv  Detail & Related papers  (2024-02-01T21:16:40Z)
• Reconstruction of Sound Field through Diffusion Models [15.192190218332843]
  Reconstructing the sound field in a room is an important task for several applications, such as sound control and augmented (AR) or virtual reality (VR) We propose a data-driven generative model for reconstructing the magnitude of acoustic fields in rooms with a focus on the modal frequency range. We introduce, for the first time, the use of a conditional Denoising Diffusion Probabilistic Model (DDPM) trained in order to reconstruct the sound field (SF-Diff) over an extended domain.
  arXiv  Detail & Related papers  (2023-12-14T11:11:26Z)
• Adaptive Shells for Efficient Neural Radiance Field Rendering [92.18962730460842]
  We propose a neural radiance formulation that smoothly transitions between- and surface-based rendering. Our approach enables efficient rendering at very high fidelity. We also demonstrate that the extracted envelope enables downstream applications such as animation and simulation.
  arXiv  Detail & Related papers  (2023-11-16T18:58:55Z)
• Generative adversarial networks with physical sound field priors [6.256923690998173]
  This paper presents a deep learning-based approach for learns-temporal reconstruction of sound fields using Generative Adversa Networks (GANs) The proposed method uses a plane wave basis and the underlying statistical distributions of pressure in rooms to reconstruct sound fields from a limited number of measurements. The results suggest that this approach provides a promising approach to sound field reconstruction using generative models that allow for a physically informed acoustics prior to problems.
  arXiv  Detail & Related papers  (2023-08-01T10:11:23Z)
• $PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction [97.06927852165464]
  Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision. We propose a novel method for single-image 3D reconstruction which generates a sparse point cloud via a conditional denoising diffusion process.
  arXiv  Detail & Related papers  (2023-02-21T13:37:07Z)
• Score-based Diffusion Models in Function Space [140.792362459734]
  Diffusion models have recently emerged as a powerful framework for generative modeling. We introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Super-resolution GANs of randomly-seeded fields [68.8204255655161]
  We propose a novel super-resolution generative adversarial network (GAN) framework to estimate field quantities from random sparse sensors. The algorithm exploits random sampling to provide incomplete views of the high-resolution underlying distributions. The proposed technique is tested on synthetic databases of fluid flow simulations, ocean surface temperature distributions measurements, and particle image velocimetry data.
  arXiv  Detail & Related papers  (2022-02-23T18:57:53Z)
• Blind Room Parameter Estimation Using Multiple-Multichannel Speech Recordings [37.145413836886455]
  Knowing the geometrical and acoustical parameters of a room may benefit applications such as audio augmented reality, speech dereverberation or audio forensics. We study the problem of jointly estimating the total surface area, the volume, as well as the frequency-dependent reverberation time and mean surface absorption of a room. A novel convolutional neural network architecture leveraging both single- and inter-channel cues is proposed and trained on a large, realistic simulated dataset.
  arXiv  Detail & Related papers  (2021-07-29T08:51:49Z)
• Volume Rendering of Neural Implicit Surfaces [57.802056954935495]
  This paper aims to improve geometry representation and reconstruction in neural volume rendering. We achieve that by modeling the volume density as a function of the geometry. Applying this new density representation to challenging scene multiview datasets produced high quality geometry reconstructions.
  arXiv  Detail & Related papers  (2021-06-22T20:23:16Z)
• Deep Sound Field Reconstruction in Real Rooms: Introducing the ISOBEL Sound Field Dataset [0.0]
  This paper extends evaluations of sound field reconstruction at low frequencies by introducing a dataset with measurements from four real rooms. The paper advances on a recent deep learning-based method for sound field reconstruction using a very low number of microphones.
  arXiv  Detail & Related papers  (2021-02-12T11:34:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.