Deep Equilibrium Models for Video Snapshot Compressive Imaging
- URL: http://arxiv.org/abs/2201.06931v1
- Date: Tue, 18 Jan 2022 12:49:59 GMT
- Title: Deep Equilibrium Models for Video Snapshot Compressive Imaging
- Authors: Yaping Zhao, Siming Zheng, Xin Yuan
- Abstract summary: We propose deep equilibrium models (DEQ) for video compressive imaging (SCI) reconstruction.
Each equilibrium model implicitly learns a nonexpansive operator and analytically computes the fixed point, thus enabling unlimited iterative steps and infinite network depth.
- Score: 7.723778852967041
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The ability of snapshot compressive imaging (SCI) systems to efficiently
capture high-dimensional (HD) data has led to an inverse problem, which
consists of recovering the HD signal from the compressed and noisy measurement.
While reconstruction algorithms grow fast to solve it with the recent advances
of deep learning, the fundamental issue of accurate and stable recovery
remains. To this end, we propose deep equilibrium models (DEQ) for video SCI,
fusing data-driven regularization and stable convergence in a theoretically
sound manner. Each equilibrium model implicitly learns a nonexpansive operator
and analytically computes the fixed point, thus enabling unlimited iterative
steps and infinite network depth with only a constant memory requirement in
training and testing. Specifically, we demonstrate how DEQ can be applied to
two existing models for video SCI reconstruction: recurrent neural networks
(RNN) and Plug-and-Play (PnP) algorithms. On a variety of datasets and real
data, both quantitative and qualitative evaluations of our results demonstrate
the effectiveness and stability of our proposed method. The code and models
will be released to the public.
Related papers
- Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms [15.54330224419903]
Snapshot compressive imaging (SCI) recovers high-dimensional (3D) data cubes from a single 2D measurement.
In this paper, we focus on SCI recovery algorithms that employ untrained neural networks (UNNs), such as deep image prior (DIP), to model source structure.
arXiv Detail & Related papers (2024-06-06T02:22:43Z) - MsDC-DEQ-Net: Deep Equilibrium Model (DEQ) with Multi-scale Dilated
Convolution for Image Compressive Sensing (CS) [0.0]
Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods.
We develop an interpretable and concise neural network model for reconstructing natural images using CS.
The model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods.
arXiv Detail & Related papers (2024-01-05T16:25:58Z) - Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance.
Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs.
In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
arXiv Detail & Related papers (2023-11-20T08:27:56Z) - Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems.
Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions.
This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
arXiv Detail & Related papers (2023-07-20T05:49:21Z) - Implicit Stochastic Gradient Descent for Training Physics-informed
Neural Networks [51.92362217307946]
Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems.
PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features.
In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
arXiv Detail & Related papers (2023-03-03T08:17:47Z) - On Robust Numerical Solver for ODE via Self-Attention Mechanism [82.95493796476767]
We explore training efficient and robust AI-enhanced numerical solvers with a small data size by mitigating intrinsic noise disturbances.
We first analyze the ability of the self-attention mechanism to regulate noise in supervised learning and then propose a simple-yet-effective numerical solver, Attr, which introduces an additive self-attention mechanism to the numerical solution of differential equations.
arXiv Detail & Related papers (2023-02-05T01:39:21Z) - Self-Supervised Deep Equilibrium Models for Inverse Problems with
Theoretical Guarantees [12.724731147883316]
DEQ models-implicit neural networks with effectively infinite number of layers-were shown to achieve state-of-the-art image reconstruction without the memory complexity associated with deep unfolding (DU)
We present self-supervised deep equilibrium model (SelfDEQ) as the first self-supervised reconstruction framework for training model-based implicit networks from undersampled and noisy MRI measurements.
arXiv Detail & Related papers (2022-10-07T22:19:26Z) - Revisit Dictionary Learning for Video Compressive Sensing under the
Plug-and-Play Framework [4.988065198958319]
In this paper, we propose an efficient and effective compressive-based algorithm for video SCI reconstruction.
Our simulation results demonstrate the effectiveness of our proposed method.
arXiv Detail & Related papers (2021-10-11T02:30:54Z) - Stabilizing Equilibrium Models by Jacobian Regularization [151.78151873928027]
Deep equilibrium networks (DEQs) are a new class of models that eschews traditional depth in favor of finding the fixed point of a single nonlinear layer.
We propose a regularization scheme for DEQ models that explicitly regularizes the Jacobian of the fixed-point update equations to stabilize the learning of equilibrium models.
We show that this regularization adds only minimal computational cost, significantly stabilizes the fixed-point convergence in both forward and backward passes, and scales well to high-dimensional, realistic domains.
arXiv Detail & Related papers (2021-06-28T00:14:11Z) - Closed-form Continuous-Depth Models [99.40335716948101]
Continuous-depth neural models rely on advanced numerical differential equation solvers.
We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
arXiv Detail & Related papers (2021-06-25T22:08:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.