Enhancement of 3D Camera Synthetic Training Data with Noise Models

• URL: http://arxiv.org/abs/2402.16514v1
• Date: Mon, 26 Feb 2024 11:50:42 GMT
• Title: Enhancement of 3D Camera Synthetic Training Data with Noise Models
• Authors: Katar\'ina Osvaldov\'a, Luk\'a\v{s} Gajdo\v{s}ech, Viktor Kocur, Martin Madaras
• Abstract summary: The goal of this paper is to assess the impact of noise in 3D camera-captured data by modeling the noise of the imaging process. We compiled a dataset of specifically constructed scenes to obtain a noise model. We specifically model lateral noise, affecting the position of captured points in the image plane, and axial noise, affecting the position along the axis to the image plane.
• Score: 1.6385815610837167
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The goal of this paper is to assess the impact of noise in 3D camera-captured data by modeling the noise of the imaging process and applying it on synthetic training data. We compiled a dataset of specifically constructed scenes to obtain a noise model. We specifically model lateral noise, affecting the position of captured points in the image plane, and axial noise, affecting the position along the axis perpendicular to the image plane. The estimated models can be used to emulate noise in synthetic training data. The added benefit of adding artificial noise is evaluated in an experiment with rendered data for object segmentation. We train a series of neural networks with varying levels of noise in the data and measure their ability to generalize on real data. The results show that using too little or too much noise can hurt the networks' performance indicating that obtaining a model of noise from real scanners is beneficial for synthetic data generation.

Related papers

• Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark [65.79402756995084]
  Real Acoustic Fields (RAF) is a new dataset that captures real acoustic room data from multiple modalities. RAF is the first dataset to provide densely captured room acoustic data.
  arXiv  Detail & Related papers  (2024-03-27T17:59:56Z)
• Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising [22.11250276261829]
  Recently, the mainstream practice for training low-light raw image denoising has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. We propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency.
  arXiv  Detail & Related papers  (2023-10-13T14:14:43Z)
• Realistic Noise Synthesis with Diffusion Models [68.48859665320828]
  Deep image denoising models often rely on large amount of training data for the high quality performance. We propose a novel method that synthesizes realistic noise using diffusion models, namely Realistic Noise Synthesize Diffusor (RNSD) RNSD can incorporate guided multiscale content, such as more realistic noise with spatial correlations can be generated at multiple frequencies.
  arXiv  Detail & Related papers  (2023-05-23T12:56:01Z)
• NoiseTransfer: Image Noise Generation with Contrastive Embeddings [9.322843611215486]
  We propose a new generative model that can synthesize noisy images with multiple different noise distributions. We adopt recent contrastive learning to learn distinguishable latent features of the noise. Our model can generate new noisy images by transferring the noise characteristics solely from a single reference noisy image.
  arXiv  Detail & Related papers  (2023-01-31T11:09:15Z)
• FSID: Fully Synthetic Image Denoising via Procedural Scene Generation [12.277286575812441]
  We develop a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks. Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations. We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results.
  arXiv  Detail & Related papers  (2022-12-07T21:21:55Z)
• Estimating Fine-Grained Noise Model via Contrastive Learning [11.626812663592416]
  We propose an innovative noise model estimation and noise synthesis pipeline for realistic noisy image generation. Our model learns a noise estimation model with fine-grained statistical noise model in a contrastive manner. By calibrating noise models of several sensors, our model can be extended to predict other cameras.
  arXiv  Detail & Related papers  (2022-04-03T02:35:01Z)
• Rethinking Noise Synthesis and Modeling in Raw Denoising [75.55136662685341]
  We introduce a new perspective to synthesize noise by directly sampling from the sensor's real noise. It inherently generates accurate raw image noise for different camera sensors.
  arXiv  Detail & Related papers  (2021-10-10T10:45:24Z)
• Physics-based Noise Modeling for Extreme Low-light Photography [63.65570751728917]
  We study the noise statistics in the imaging pipeline of CMOS photosensors. We formulate a comprehensive noise model that can accurately characterize the real noise structures. Our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms.
  arXiv  Detail & Related papers  (2021-08-04T16:36:29Z)
• Adaptive noise imitation for image denoising [58.21456707617451]
  We develop a new textbfadaptive noise imitation (ADANI) algorithm that can synthesize noisy data from naturally noisy images. To produce realistic noise, a noise generator takes unpaired noisy/clean images as input, where the noisy image is a guide for noise generation. Coupling the noisy data output from ADANI with the corresponding ground-truth, a denoising CNN is then trained in a fully-supervised manner.
  arXiv  Detail & Related papers  (2020-11-30T02:49:36Z)
• A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising [34.98772175073111]
  We present a highly accurate noise formation model based on the characteristics of CMOS photosensors. We also propose a method to calibrate the noise parameters for available modern digital cameras.
  arXiv  Detail & Related papers  (2020-03-28T09:16:48Z)
• CycleISP: Real Image Restoration via Improved Data Synthesis [166.17296369600774]
  We present a framework that models camera imaging pipeline in forward and reverse directions. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-17T15:20:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.