Equilibrated Zeroth-Order Unrolled Deep Networks for Accelerated MRI
- URL: http://arxiv.org/abs/2112.09891v1
- Date: Sat, 18 Dec 2021 09:47:19 GMT
- Title: Equilibrated Zeroth-Order Unrolled Deep Networks for Accelerated MRI
- Authors: Zhuo-Xu Cui, Jing Cheng, Qinyong Zhu, Yuanyuan Liu, Sen Jia, Kankan
Zhao, Ziwen Ke, Wenqi Huang, Haifeng Wang, Yanjie Zhu, Dong Liang
- Abstract summary: Recently, model-driven deep learning unrolls a certain iterative algorithm of a regularization model into a cascade network.
In theory, there is not necessarily such a functional regularizer whose first-order information matches the replaced network module.
This paper propose to present a safeguarded methodology on network unrolling.
- Score: 14.586911990418624
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Recently, model-driven deep learning unrolls a certain iterative algorithm of
a regularization model into a cascade network by replacing the first-order
information (i.e., (sub)gradient or proximal operator) of the regularizer with
a network module, which appears more explainable and predictable compared to
common data-driven networks. Conversely, in theory, there is not necessarily
such a functional regularizer whose first-order information matches the
replaced network module, which means the network output may not be covered by
the original regularization model. Moreover, up to now, there is also no theory
to guarantee the global convergence and robustness (regularity) of unrolled
networks under realistic assumptions. To bridge this gap, this paper propose to
present a safeguarded methodology on network unrolling. Specifically, focusing
on accelerated MRI, we unroll a zeroth-order algorithm, of which the network
module represents the regularizer itself, so that the network output can be
still covered by the regularization model. Furthermore, inspired by the ideal
of deep equilibrium models, before backpropagating, we carry out the unrolled
iterative network to converge to a fixed point to ensure the convergence. In
case the measurement data contains noise, we prove that the proposed network is
robust against noisy interference. Finally, numerical experiments show that the
proposed network consistently outperforms the state-of-the-art MRI
reconstruction methods including traditional regularization methods and other
deep learning methods.
Related papers
- Matrix Completion-Informed Deep Unfolded Equilibrium Models for
Self-Supervised k-Space Interpolation in MRI [8.33626757808923]
Regularization model-driven deep learning (DL) has gained significant attention due to its ability to leverage the potent representational capabilities of DL.
We propose a self-supervised DL approach for accelerated MRI that is theoretically guaranteed and does not rely on fully sampled labels.
arXiv Detail & Related papers (2023-09-24T07:25:06Z) - Robust Training and Verification of Implicit Neural Networks: A
Non-Euclidean Contractive Approach [64.23331120621118]
This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks.
We introduce a related embedded network and show that the embedded network can be used to provide an $ell_infty$-norm box over-approximation of the reachable sets of the original network.
We apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.
arXiv Detail & Related papers (2022-08-08T03:13:24Z) - Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model.
A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations.
We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
arXiv Detail & Related papers (2022-06-08T04:09:13Z) - Deep Convolutional Learning-Aided Detector for Generalized Frequency
Division Multiplexing with Index Modulation [0.0]
The proposed method first pre-processes the received signal by using a zero-forcing (ZF) detector and then uses a neural network consisting of a convolutional neural network (CNN) followed by a fully-connected neural network (FCNN)
The FCNN part uses only two fully-connected layers, which can be adapted to yield a trade-off between complexity and bit error rate (BER) performance.
It has been demonstrated that the proposed deep convolutional neural network-based detection and demodulation scheme provides better BER performance compared to ZF detector with a reasonable complexity increase.
arXiv Detail & Related papers (2022-02-06T22:18:42Z) - Robustness Certificates for Implicit Neural Networks: A Mixed Monotone
Contractive Approach [60.67748036747221]
Implicit neural networks offer competitive performance and reduced memory consumption.
They can remain brittle with respect to input adversarial perturbations.
This paper proposes a theoretical and computational framework for robustness verification of implicit neural networks.
arXiv Detail & Related papers (2021-12-10T03:08:55Z) - Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for
sparse recover [87.28082715343896]
We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications.
We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem.
The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
arXiv Detail & Related papers (2021-10-20T06:15:45Z) - Algorithm Unrolling for Massive Access via Deep Neural Network with
Theoretical Guarantee [30.86806523281873]
Massive access is a critical design challenge of Internet of Things (IoT) networks.
We consider the grant-free uplink transmission of an IoT network with a multiple-antenna base station (BS) and a large number of single-antenna IoT devices.
We propose a novel algorithm unrolling framework based on the deep neural network to simultaneously achieve low computational complexity and high robustness.
arXiv Detail & Related papers (2021-06-19T05:23:05Z) - Improve Generalization and Robustness of Neural Networks via Weight
Scale Shifting Invariant Regularizations [52.493315075385325]
We show that a family of regularizers, including weight decay, is ineffective at penalizing the intrinsic norms of weights for networks with homogeneous activation functions.
We propose an improved regularizer that is invariant to weight scale shifting and thus effectively constrains the intrinsic norm of a neural network.
arXiv Detail & Related papers (2020-08-07T02:55:28Z) - Unsupervised Adaptive Neural Network Regularization for Accelerated
Radial Cine MRI [3.6280929178575994]
We propose an iterative reconstruction scheme for 2D radial cine MRI based on ground truth-free unsupervised learning of shallow convolutional neural networks.
The network is trained to approximate patches of the current estimate of the solution during the reconstruction.
arXiv Detail & Related papers (2020-02-10T14:47:20Z) - MSE-Optimal Neural Network Initialization via Layer Fusion [68.72356718879428]
Deep neural networks achieve state-of-the-art performance for a range of classification and inference tasks.
The use of gradient combined nonvolutionity renders learning susceptible to novel problems.
We propose fusing neighboring layers of deeper networks that are trained with random variables.
arXiv Detail & Related papers (2020-01-28T18:25:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.