Unsupervised Training of Convex Regularizers using Maximum Likelihood Estimation

• URL: http://arxiv.org/abs/2404.05445v2
• Date: Mon, 29 Jul 2024 12:10:27 GMT
• Title: Unsupervised Training of Convex Regularizers using Maximum Likelihood Estimation
• Authors: Hong Ye Tan, Ziruo Cai, Marcelo Pereyra, Subhadip Mukherjee, Junqi Tang, Carola-Bibiane Schönlieb,
• Abstract summary: We propose an unsupervised approach using maximum marginal likelihood estimation to train a convex neural network-based image regularization term directly on noisy measurements. Experiments demonstrate that the proposed method produces priors that are near competitive when compared to the analogous supervised training method for various image corruption operators.
• Score: 12.625383613718636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imaging is a standard example of an inverse problem, where the task of reconstructing a ground truth from a noisy measurement is ill-posed. Recent state-of-the-art approaches for imaging use deep learning, spearheaded by unrolled and end-to-end models and trained on various image datasets. However, many such methods require the availability of ground truth data, which may be unavailable or expensive, leading to a fundamental barrier that can not be bypassed by choice of architecture. Unsupervised learning presents an alternative paradigm that bypasses this requirement, as they can be learned directly on noisy data and do not require any ground truths. A principled Bayesian approach to unsupervised learning is to maximize the marginal likelihood with respect to the given noisy measurements, which is intrinsically linked to classical variational regularization. We propose an unsupervised approach using maximum marginal likelihood estimation to train a convex neural network-based image regularization term directly on noisy measurements, improving upon previous work in both model expressiveness and dataset size. Experiments demonstrate that the proposed method produces priors that are near competitive when compared to the analogous supervised training method for various image corruption operators, maintaining significantly better generalization properties when compared to end-to-end methods. Moreover, we provide a detailed theoretical analysis of the convergence properties of our proposed algorithm.

Related papers

• Learning Diffusion Model from Noisy Measurement using Principled Expectation-Maximization Method [9.173055778539641]
  We propose a principled expectation-maximization (EM) framework that iteratively learns diffusion models from noisy data with arbitrary corruption types. Our framework employs a plug-and-play Monte Carlo method to accurately estimate clean images from noisy measurements, followed by training the diffusion model using the reconstructed images.
  arXiv  Detail & Related papers  (2024-10-15T03:54:59Z)
• Generative Edge Detection with Stable Diffusion [52.870631376660924]
  Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. We propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. We conduct extensive experiments on multiple datasets and achieve competitive performance.
  arXiv  Detail & Related papers  (2024-10-04T01:52:23Z)
• Adversarial Robustification via Text-to-Image Diffusion Models [56.37291240867549]
  Adrial robustness has been conventionally believed as a challenging property to encode for neural networks. We develop a scalable and model-agnostic solution to achieve adversarial robustness without using any data.
  arXiv  Detail & Related papers  (2024-07-26T10:49:14Z)
• Consistency Regularisation for Unsupervised Domain Adaptation in Monocular Depth Estimation [15.285720572043678]
  We formulate unsupervised domain adaptation for monocular depth estimation as a consistency-based semi-supervised learning problem. We introduce a pairwise loss function that regularises predictions on the source domain while enforcing consistency across multiple augmented views. In our experiments, we rely on the standard depth estimation benchmarks KITTI and NYUv2 to demonstrate state-of-the-art results.
  arXiv  Detail & Related papers  (2024-05-27T23:32:06Z)
• Model-based Offline Imitation Learning with Non-expert Data [7.615595533111191]
  We propose a scalable model-based offline imitation learning algorithmic framework that leverages datasets collected by both suboptimal and optimal policies. We show that the proposed method textitalways outperforms Behavioral Cloning in the low data regime on simulated continuous control domains.
  arXiv  Detail & Related papers  (2022-06-11T13:08:08Z)
• PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization [57.37911115888587]
  We introduce a regularizer for the variational modeling of inverse problems in imaging based on normalizing flows. Our regularizer, called patchNR, involves a normalizing flow learned on patches of very few images.
  arXiv  Detail & Related papers  (2022-05-24T12:14:26Z)
• Deblurring via Stochastic Refinement [85.42730934561101]
  We present an alternative framework for blind deblurring based on conditional diffusion models. Our method is competitive in terms of distortion metrics such as PSNR.
  arXiv  Detail & Related papers  (2021-12-05T04:36:09Z)
• Unsupervised Source Separation via Bayesian Inference in the Latent Domain [4.583433328833251]
  State of the art audio source separation models rely on supervised data-driven approaches. We propose a simple yet effective unsupervised separation algorithm, which operates directly on a latent representation of time-domain signals. We validate our approach on the Slakh dataset arXiv:1909.08494, demonstrating results in line with state of the art supervised approaches.
  arXiv  Detail & Related papers  (2021-10-11T14:32:55Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Learning regularization and intensity-gradient-based fidelity for single image super resolution [0.0]
  We study the image degradation progress, and establish degradation model both in intensity and gradient space. A comprehensive data consistency constraint is established for the reconstruction. The proposed fidelity term and designed regularization term are embedded into the regularization framework.
  arXiv  Detail & Related papers  (2020-03-24T07:03:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.