What's Behind the Mask: Estimating Uncertainty in Image-to-Image Problems

• URL: http://arxiv.org/abs/2211.15211v1
• Date: Mon, 28 Nov 2022 10:41:34 GMT
• Title: What's Behind the Mask: Estimating Uncertainty in Image-to-Image Problems
• Authors: Gilad Kutiel, Regev Cohen, Michael Elad, Daniel Freedman
• Abstract summary: Estimating uncertainty in image-to-image networks is an important task. We introduce a new approach to this problem based on masking. We evaluate our mask-based approach to uncertainty on image colorization, image completion, and super-resolution tasks.
• Score: 29.55144604924458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating uncertainty in image-to-image networks is an important task, particularly as such networks are being increasingly deployed in the biological and medical imaging realms. In this paper, we introduce a new approach to this problem based on masking. Given an existing image-to-image network, our approach computes a mask such that the distance between the masked reconstructed image and the masked true image is guaranteed to be less than a specified threshold, with high probability. The mask thus identifies the more certain regions of the reconstructed image. Our approach is agnostic to the underlying image-to-image network, and only requires triples of the input (degraded), reconstructed and true images for training. Furthermore, our method is agnostic to the distance metric used. As a result, one can use $L_p$-style distances or perceptual distances like LPIPS, which contrasts with interval-based approaches to uncertainty. Our theoretical guarantees derive from a conformal calibration procedure. We evaluate our mask-based approach to uncertainty on image colorization, image completion, and super-resolution tasks, demonstrating high quality performance on each.

Related papers

• Variance-insensitive and Target-preserving Mask Refinement for Interactive Image Segmentation [68.16510297109872]
  Point-based interactive image segmentation can ease the burden of mask annotation in applications such as semantic segmentation and image editing. We introduce a novel method, Variance-Insensitive and Target-Preserving Mask Refinement to enhance segmentation quality with fewer user inputs. Experiments on GrabCut, Berkeley, SBD, and DAVIS datasets demonstrate our method's state-of-the-art performance in interactive image segmentation.
  arXiv  Detail & Related papers  (2023-12-22T02:31:31Z)
• On Mask-based Image Set Desensitization with Recognition Support [46.51027529020668]
  We propose a mask-based image desensitization approach while supporting recognition. We exploit an interpretation algorithm to maintain critical information for the recognition task. In addition, we propose a feature selection masknet as the model adjustment method to improve the performance based on the masked images.
  arXiv  Detail & Related papers  (2023-12-14T14:26:42Z)
• DPPMask: Masked Image Modeling with Determinantal Point Processes [49.65141962357528]
  Masked Image Modeling (MIM) has achieved impressive representative performance with the aim of reconstructing randomly masked images. We show that uniformly random masking widely used in previous works unavoidably loses some key objects and changes original semantic information. To address this issue, we augment MIM with a new masking strategy namely the DPPMask. Our method is simple yet effective and requires no extra learnable parameters when implemented within various frameworks.
  arXiv  Detail & Related papers  (2023-03-13T13:40:39Z)
• Improving Masked Autoencoders by Learning Where to Mask [65.89510231743692]
  Masked image modeling is a promising self-supervised learning method for visual data. We present AutoMAE, a framework that uses Gumbel-Softmax to interlink an adversarially-trained mask generator and a mask-guided image modeling process. In our experiments, AutoMAE is shown to provide effective pretraining models on standard self-supervised benchmarks and downstream tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:28:55Z)
• Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps [85.67745220834718]
  We present an edge-guided learnable bidirectional attention map (Edge-LBAM) for improving image inpainting of irregular holes. Our Edge-LBAM method contains dual procedures,including structure-aware mask-updating guided by predict edges. Extensive experiments show that our Edge-LBAM is effective in generating coherent image structures and preventing color discrepancy and blurriness.
  arXiv  Detail & Related papers  (2021-04-25T07:25:16Z)
• OLED: One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection [1.933681537640272]
  novelty detection is the task of recognizing samples that do not belong to the distribution of the target class. Deep autoencoders have been widely used as a base of many unsupervised novelty detection methods. We have designed a framework consisting of two competing networks, a Mask Module and a Reconstructor.
  arXiv  Detail & Related papers  (2021-03-27T17:59:40Z)
• TextureWGAN: Texture Preserving WGAN with MLE Regularizer for Inverse Problems [4.112614964808275]
  Among all proposed methods, the most popular and effective method is the convolutional neural network (CNN) with mean square error (MSE) We proposed a new method based on Wasserstein GAN (WGAN) for inverse problems. We showed that the WGAN-based method was effective to preserve image texture. It also used a maximum likelihood estimation (MLE) regularizer to preserve pixel fidelity.
  arXiv  Detail & Related papers  (2020-08-11T17:06:34Z)
• R-MNet: A Perceptual Adversarial Network for Image Inpainting [5.471225956329675]
  We propose a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting. We show that our method is able to generalize to high-resolution inpainting task, and further show more realistic outputs that are plausible to the human visual system.
  arXiv  Detail & Related papers  (2020-08-11T10:58:10Z)
• Exploiting Semantics for Face Image Deblurring [121.44928934662063]
  We propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks. We incorporate face semantic labels as input priors and propose an adaptive structural loss to regularize facial local structures. The proposed method restores sharp images with more accurate facial features and details.
  arXiv  Detail & Related papers  (2020-01-19T13:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.