FSID: Fully Synthetic Image Denoising via Procedural Scene Generation

• URL: http://arxiv.org/abs/2212.03961v1
• Date: Wed, 7 Dec 2022 21:21:55 GMT
• Title: FSID: Fully Synthetic Image Denoising via Procedural Scene Generation
• Authors: Gyeongmin Choe, Beibei Du, Seonghyeon Nam, Xiaoyu Xiang, Bo Zhu, Rakesh Ranjan
• Abstract summary: We develop a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks. Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations. We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results.
• Score: 12.277286575812441
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For low-level computer vision and image processing ML tasks, training on large datasets is critical for generalization. However, the standard practice of relying on real-world images primarily from the Internet comes with image quality, scalability, and privacy issues, especially in commercial contexts. To address this, we have developed a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks. Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations. Then, we calibrate the camera noise profiles to synthesize the noisy images. From this pipeline, we generated a fully synthetic image denoising dataset (FSID) which consists of 175,000 noisy/clean image pairs. We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results when evaluated on real-world noisy images captured with smartphone cameras.

Related papers

• 3D Human Reconstruction in the Wild with Synthetic Data Using Generative Models [52.96248836582542]
  We propose an effective approach based on recent diffusion models, termed HumanWild, which can effortlessly generate human images and corresponding 3D mesh annotations. By exclusively employing generative models, we generate large-scale in-the-wild human images and high-quality annotations, eliminating the need for real-world data collection.
  arXiv  Detail & Related papers  (2024-03-17T06:31:16Z)
• Enhancement of 3D Camera Synthetic Training Data with Noise Models [1.6385815610837167]
  The goal of this paper is to assess the impact of noise in 3D camera-captured data by modeling the noise of the imaging process. We compiled a dataset of specifically constructed scenes to obtain a noise model. We specifically model lateral noise, affecting the position of captured points in the image plane, and axial noise, affecting the position along the axis to the image plane.
  arXiv  Detail & Related papers  (2024-02-26T11:50:42Z)
• Synfeal: A Data-Driven Simulator for End-to-End Camera Localization [0.9749560288448114]
  We propose a framework that synthesizes large localization datasets based on realistic 3D reconstructions of the real world. Our framework, Synfeal, is an open-source, data-driven simulator that synthesizes RGB images by moving a virtual camera through a realistic 3D textured mesh. The results validate that the training of camera localization algorithms on datasets generated by Synfeal leads to better results when compared to datasets generated by state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-29T17:29:02Z)
• Is synthetic data from generative models ready for image recognition? [69.42645602062024]
  We study whether and how synthetic images generated from state-of-the-art text-to-image generation models can be used for image recognition tasks. We showcase the powerfulness and shortcomings of synthetic data from existing generative models, and propose strategies for better applying synthetic data for recognition tasks.
  arXiv  Detail & Related papers  (2022-10-14T06:54:24Z)
• Towards Real-World Video Deblurring by Exploring Blur Formation Process [53.91239555063343]
  In recent years, deep learning-based approaches have achieved promising success on video deblurring task. The models trained on existing synthetic datasets still suffer from generalization problems over real-world blurry scenarios. We propose a novel realistic blur synthesis pipeline termed RAW-Blur by leveraging blur formation cues.
  arXiv  Detail & Related papers  (2022-08-28T09:24:52Z)
• Leveraging Deepfakes to Close the Domain Gap between Real and Synthetic Images in Facial Capture Pipelines [8.366597450893456]
  We propose an end-to-end pipeline for building and tracking 3D facial models from personalized in-the-wild video data. We present a method for automatic data curation and retrieval based on a hierarchical clustering framework typical of collision algorithms in traditional computer graphics pipelines. We outline how we train a motion capture regressor, leveraging the aforementioned techniques to avoid the need for real-world ground truth data.
  arXiv  Detail & Related papers  (2022-04-22T15:09:49Z)
• IDR: Self-Supervised Image Denoising via Iterative Data Refinement [66.5510583957863]
  We present a practical unsupervised image denoising method to achieve state-of-the-art denoising performance. Our method only requires single noisy images and a noise model, which is easily accessible in practical raw image denoising. To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
  arXiv  Detail & Related papers  (2021-11-29T07:22:53Z)
• Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy [22.566600256820646]
  We propose a simulation-based denoising (SBD) framework, in which CNNs are trained on simulated images. SBD outperforms existing techniques by a wide margin on a simulated benchmark dataset, as well as on real data. We release the first publicly available benchmark dataset of TEM images, containing 18,000 examples.
  arXiv  Detail & Related papers  (2020-10-24T19:59:28Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)
• CycleISP: Real Image Restoration via Improved Data Synthesis [166.17296369600774]
  We present a framework that models camera imaging pipeline in forward and reverse directions. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-17T15:20:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.