UMat: Uncertainty-Aware Single Image High Resolution Material Capture
- URL: http://arxiv.org/abs/2305.16312v1
- Date: Thu, 25 May 2023 17:59:04 GMT
- Title: UMat: Uncertainty-Aware Single Image High Resolution Material Capture
- Authors: Carlos Rodriguez-Pardo, Henar Dominguez-Elvira, David
Pascual-Hernandez, Elena Garces
- Abstract summary: We propose a learning-based method to recover normals, specularity, and roughness from a single diffuse image of a material.
Our method is the first one to deal with the problem of modeling uncertainty in material digitization.
- Score: 2.416160525187799
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We propose a learning-based method to recover normals, specularity, and
roughness from a single diffuse image of a material, using microgeometry
appearance as our primary cue. Previous methods that work on single images tend
to produce over-smooth outputs with artifacts, operate at limited resolution,
or train one model per class with little room for generalization. Previous
methods that work on single images tend to produce over-smooth outputs with
artifacts, operate at limited resolution, or train one model per class with
little room for generalization. In contrast, in this work, we propose a novel
capture approach that leverages a generative network with attention and a U-Net
discriminator, which shows outstanding performance integrating global
information at reduced computational complexity. We showcase the performance of
our method with a real dataset of digitized textile materials and show that a
commodity flatbed scanner can produce the type of diffuse illumination required
as input to our method. Additionally, because the problem might be illposed
-more than a single diffuse image might be needed to disambiguate the specular
reflection- or because the training dataset is not representative enough of the
real distribution, we propose a novel framework to quantify the model's
confidence about its prediction at test time. Our method is the first one to
deal with the problem of modeling uncertainty in material digitization,
increasing the trustworthiness of the process and enabling more intelligent
strategies for dataset creation, as we demonstrate with an active learning
experiment.
Related papers
- DiffUHaul: A Training-Free Method for Object Dragging in Images [78.93531472479202]
We propose a training-free method, dubbed DiffUHaul, for the object dragging task.
We first apply attention masking in each denoising step to make the generation more disentangled across different objects.
In the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance.
arXiv Detail & Related papers (2024-06-03T17:59:53Z) - Learning from small data sets: Patch-based regularizers in inverse
problems for image reconstruction [1.1650821883155187]
Recent advances in machine learning require a huge amount of data and computer capacity to train the networks.
Our paper addresses the issue of learning from small data sets by taking patches of very few images into account.
We show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods.
arXiv Detail & Related papers (2023-12-27T15:30:05Z) - Intrinsic Image Diffusion for Indoor Single-view Material Estimation [55.276815106443976]
We present Intrinsic Image Diffusion, a generative model for appearance decomposition of indoor scenes.
Given a single input view, we sample multiple possible material explanations represented as albedo, roughness, and metallic maps.
Our method produces significantly sharper, more consistent, and more detailed materials, outperforming state-of-the-art methods by $1.5dB$ on PSNR and by $45%$ better FID score on albedo prediction.
arXiv Detail & Related papers (2023-12-19T15:56:19Z) - Meta-Prior: Meta learning for Adaptive Inverse Problem Solvers [9.364509804053275]
Real-world imaging challenges often lack ground truth data, rendering traditional supervised approaches ineffective.
Our method trains a meta-model on a diverse set of imaging tasks that allows the model to be efficiently fine-tuned for specific tasks.
In simple settings, this approach recovers the Bayes optimal estimator, illustrating the soundness of our approach.
arXiv Detail & Related papers (2023-11-30T17:02:27Z) - Detecting Generated Images by Real Images Only [64.12501227493765]
Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training.
This paper approaches the generated image detection problem from a new perspective: Start from real images.
By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
arXiv Detail & Related papers (2023-11-02T03:09:37Z) - Free-ATM: Exploring Unsupervised Learning on Diffusion-Generated Images
with Free Attention Masks [64.67735676127208]
Text-to-image diffusion models have shown great potential for benefiting image recognition.
Although promising, there has been inadequate exploration dedicated to unsupervised learning on diffusion-generated images.
We introduce customized solutions by fully exploiting the aforementioned free attention masks.
arXiv Detail & Related papers (2023-08-13T10:07:46Z) - Disentangled Pre-training for Image Matting [74.10407744483526]
Image matting requires high-quality pixel-level human annotations to support the training of a deep model.
We propose a self-supervised pre-training approach that can leverage infinite numbers of data to boost the matting performance.
arXiv Detail & Related papers (2023-04-03T08:16:02Z) - Person Image Synthesis via Denoising Diffusion Model [116.34633988927429]
We show how denoising diffusion models can be applied for high-fidelity person image synthesis.
Our results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios.
arXiv Detail & Related papers (2022-11-22T18:59:50Z) - Rich Feature Distillation with Feature Affinity Module for Efficient
Image Dehazing [1.1470070927586016]
This work introduces a simple, lightweight, and efficient framework for single-image haze removal.
We exploit rich "dark-knowledge" information from a lightweight pre-trained super-resolution model via the notion of heterogeneous knowledge distillation.
Our experiments are carried out on the RESIDE-Standard dataset to demonstrate the robustness of our framework to the synthetic and real-world domains.
arXiv Detail & Related papers (2022-07-13T18:32:44Z) - Single Image Internal Distribution Measurement Using Non-Local
Variational Autoencoder [11.985083962982909]
This paper proposes a novel image-specific solution, namely non-local variational autoencoder (textttNLVAE)
textttNLVAE is introduced as a self-supervised strategy that reconstructs high-resolution images using disentangled information from the non-local neighbourhood.
Experimental results from seven benchmark datasets demonstrate the effectiveness of the textttNLVAE model.
arXiv Detail & Related papers (2022-04-02T18:43:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.