One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns

• URL: http://arxiv.org/abs/2404.16292v1
• Date: Thu, 25 Apr 2024 02:23:11 GMT
• Title: One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns
• Authors: Arman Maesumi, Dylan Hu, Krishi Saripalli, Vladimir G. Kim, Matthew Fisher, Sören Pirk, Daniel Ritchie,
• Abstract summary: We present a single generative model which can learn to generate multiple types of noise as well as blend between them. We also present an application of our model to improving inverse procedural material design.
• Score: 33.293193191683145
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Procedural noise is a fundamental component of computer graphics pipelines, offering a flexible way to generate textures that exhibit "natural" random variation. Many different types of noise exist, each produced by a separate algorithm. In this paper, we present a single generative model which can learn to generate multiple types of noise as well as blend between them. In addition, it is capable of producing spatially-varying noise blends despite not having access to such data for training. These features are enabled by training a denoising diffusion model using a novel combination of data augmentation and network conditioning techniques. Like procedural noise generators, the model's behavior is controllable via interpretable parameters and a source of randomness. We use our model to produce a variety of visually compelling noise textures. We also present an application of our model to improving inverse procedural material design; using our model in place of fixed-type noise nodes in a procedural material graph results in higher-fidelity material reconstructions without needing to know the type of noise in advance.

Related papers

• Bayesian Inference of General Noise Model Parameters from Surface Code's Syndrome Statistics [0.0]
  We propose general noise model Bayesian inference methods that integrate the surface code's tensor network simulator. For stationary noise, where the noise parameters are constant and do not change, we propose a method based on the Markov chain Monte Carlo. For time-varying noise, which is a more realistic situation, we introduce another method based on the sequential Monte Carlo.
  arXiv  Detail & Related papers  (2024-06-13T10:26:04Z)
• Blue noise for diffusion models [50.99852321110366]
  We introduce a novel and general class of diffusion models taking correlated noise within and across images into account. Our framework allows introducing correlation across images within a single mini-batch to improve gradient flow. We perform both qualitative and quantitative evaluations on a variety of datasets using our method.
  arXiv  Detail & Related papers  (2024-02-07T14:59:25Z)
• Can We Transfer Noise Patterns? A Multi-environment Spectrum Analysis Model Using Generated Cases [10.876490928902838]
  spectral data-based testing devices suffer from complex noise patterns when deployed in non-laboratory environments. We propose a noise patterns transferring model, which takes the spectrum of standard water samples in different environments as cases and learns the differences in their noise patterns. We generate a sample-to-sample case-base to exclude the interference of sample-level noise on dataset-level noise learning.
  arXiv  Detail & Related papers  (2023-08-02T13:29:31Z)
• Realistic Noise Synthesis with Diffusion Models [68.48859665320828]
  Deep image denoising models often rely on large amount of training data for the high quality performance. We propose a novel method that synthesizes realistic noise using diffusion models, namely Realistic Noise Synthesize Diffusor (RNSD) RNSD can incorporate guided multiscale content, such as more realistic noise with spatial correlations can be generated at multiple frequencies.
  arXiv  Detail & Related papers  (2023-05-23T12:56:01Z)
• NoiseTransfer: Image Noise Generation with Contrastive Embeddings [9.322843611215486]
  We propose a new generative model that can synthesize noisy images with multiple different noise distributions. We adopt recent contrastive learning to learn distinguishable latent features of the noise. Our model can generate new noisy images by transferring the noise characteristics solely from a single reference noisy image.
  arXiv  Detail & Related papers  (2023-01-31T11:09:15Z)
• Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training [50.018580462619425]
  We propose a novel framework, namely Pixel-level Noise-aware Generative Adrial Network (PNGAN) PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space. For better noise fitting, we present an efficient architecture Simple Multi-versa-scale Network (SMNet) as the generator.
  arXiv  Detail & Related papers  (2022-04-06T14:09:02Z)
• C2N: Practical Generative Noise Modeling for Real-World Denoising [53.96391787869974]
  We introduce a Clean-to-Noisy image generation framework, namely C2N, to imitate complex real-world noise without using paired examples. We construct the noise generator in C2N accordingly with each component of real-world noise characteristics to express a wide range of noise accurately.
  arXiv  Detail & Related papers  (2022-02-19T05:53:46Z)
• Multiview point cloud registration with anisotropic and space-varying localization noise [1.5499426028105903]
  We address the problem of registering multiple point clouds corrupted with high anisotropic localization noise. Existing methods are based on an implicit assumption of space-invariant isotropic noise. We show that our noise handling strategy improves significantly the robustness to high levels of anisotropic noise.
  arXiv  Detail & Related papers  (2022-01-03T15:21:24Z)
• Rethinking Noise Synthesis and Modeling in Raw Denoising [75.55136662685341]
  We introduce a new perspective to synthesize noise by directly sampling from the sensor's real noise. It inherently generates accurate raw image noise for different camera sensors.
  arXiv  Detail & Related papers  (2021-10-10T10:45:24Z)
• Noise Estimation for Generative Diffusion Models [91.22679787578438]
  In this work, we present a simple and versatile learning scheme that can adjust the noise parameters for any given number of steps. Our approach comes at a negligible computation cost.
  arXiv  Detail & Related papers  (2021-04-06T15:46:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.