A Residual Solver and Its Unfolding Neural Network for Total Variation
Regularized Models
- URL: http://arxiv.org/abs/2009.03477v1
- Date: Tue, 8 Sep 2020 01:44:34 GMT
- Title: A Residual Solver and Its Unfolding Neural Network for Total Variation
Regularized Models
- Authors: Yuanhao Gong
- Abstract summary: This paper proposes to solve the Total Variation regularized models by finding the residual between the input and the unknown optimal solution.
We numerically confirm that the residual solver can reach the same global optimal solutions as the classical method on 500 natural images.
Both the proposed algorithm and neural network are successfully applied on several problems to demonstrate their effectiveness and efficiency.
- Score: 5.9622541907827875
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper proposes to solve the Total Variation regularized models by
finding the residual between the input and the unknown optimal solution. After
analyzing a previous method, we developed a new iterative algorithm, named as
Residual Solver, which implicitly solves the model in gradient domain. We
theoretically prove the uniqueness of the gradient field in our algorithm. We
further numerically confirm that the residual solver can reach the same global
optimal solutions as the classical method on 500 natural images. Moreover, we
unfold our iterative algorithm into a convolution neural network (named as
Residual Solver Network). This network is unsupervised and can be considered as
an "enhanced version" of our iterative algorithm. Finally, both the proposed
algorithm and neural network are successfully applied on several problems to
demonstrate their effectiveness and efficiency, including image smoothing,
denoising, and biomedical image reconstruction. The proposed network is general
and can be applied to solve other total variation regularized models.
Related papers
- Solving Linear Inverse Problems Provably via Posterior Sampling with
Latent Diffusion Models [98.95988351420334]
We present the first framework to solve linear inverse problems leveraging pre-trained latent diffusion models.
We theoretically analyze our algorithm showing provable sample recovery in a linear model setting.
arXiv Detail & Related papers (2023-07-02T17:21:30Z) - An Optimization-based Deep Equilibrium Model for Hyperspectral Image
Deconvolution with Convergence Guarantees [71.57324258813675]
We propose a novel methodology for addressing the hyperspectral image deconvolution problem.
A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network.
The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
arXiv Detail & Related papers (2023-06-10T08:25:16Z) - AskewSGD : An Annealed interval-constrained Optimisation method to train
Quantized Neural Networks [12.229154524476405]
We develop a new algorithm, Annealed Skewed SGD - AskewSGD - for training deep neural networks (DNNs) with quantized weights.
Unlike algorithms with active sets and feasible directions, AskewSGD avoids projections or optimization under the entire feasible set.
Experimental results show that the AskewSGD algorithm performs better than or on par with state of the art methods in classical benchmarks.
arXiv Detail & Related papers (2022-11-07T18:13:44Z) - An Inexact Augmented Lagrangian Algorithm for Training Leaky ReLU Neural
Network with Group Sparsity [13.27709100571336]
A leaky ReLU network with a group regularization term has been widely used in the recent years.
We show that there is a lack of approaches to compute a stationary point deterministically.
We propose an inexact augmented Lagrangian algorithm for solving the new model.
arXiv Detail & Related papers (2022-05-11T11:53:15Z) - Regularized Training of Intermediate Layers for Generative Models for
Inverse Problems [9.577509224534323]
We introduce a principle that if a generative model is intended for inversion using an algorithm based on optimization of intermediate layers, it should be trained in a way that regularizes those intermediate layers.
We instantiate this principle for two notable recent inversion algorithms: Intermediate Layer Optimization and the Multi-Code GAN prior.
For both of these inversion algorithms, we introduce a new regularized GAN training algorithm and demonstrate that the learned generative model results in lower reconstruction errors across a wide range of under sampling ratios.
arXiv Detail & Related papers (2022-03-08T20:30:49Z) - Message Passing Neural PDE Solvers [60.77761603258397]
We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators.
We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes.
We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
arXiv Detail & Related papers (2022-02-07T17:47:46Z) - Fractal Structure and Generalization Properties of Stochastic
Optimization Algorithms [71.62575565990502]
We prove that the generalization error of an optimization algorithm can be bounded on the complexity' of the fractal structure that underlies its generalization measure.
We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden/layered neural networks) and algorithms.
arXiv Detail & Related papers (2021-06-09T08:05:36Z) - Accuracy and Architecture Studies of Residual Neural Network solving
Ordinary Differential Equations [0.0]
We consider utilizing a residual neural network (ResNet) to solve ordinary differential equations.
We apply forward Euler, Runge-Kutta2 and Runge-Kutta4 finite difference methods to generate three sets of targets training the ResNet.
The well trained ResNet behaves just as its counterpart of the corresponding one-step finite difference method.
arXiv Detail & Related papers (2021-01-10T17:34:10Z) - Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
Blind image restoration is a common yet challenging problem in computer vision.
We propose a novel blind image restoration method, aiming to integrate both the advantages of them.
Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
arXiv Detail & Related papers (2020-08-25T03:30:53Z) - Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction [4.2476585678737395]
We propose a general learning based framework for solving nonsmooth non image reconstruction problems.
We show that the proposed is-efficient convergence state-of-the-art methods in an image problems in training.
arXiv Detail & Related papers (2020-07-22T07:59:07Z) - GACEM: Generalized Autoregressive Cross Entropy Method for Multi-Modal
Black Box Constraint Satisfaction [69.94831587339539]
We present a modified Cross-Entropy Method (CEM) that uses a masked auto-regressive neural network for modeling uniform distributions over the solution space.
Our algorithm is able to express complicated solution spaces, thus allowing it to track a variety of different solution regions.
arXiv Detail & Related papers (2020-02-17T20:21:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.