Operator Sketching for Deep Unrolling Networks

• URL: http://arxiv.org/abs/2203.11156v2
• Date: Tue, 22 Mar 2022 10:25:45 GMT
• Title: Operator Sketching for Deep Unrolling Networks
• Authors: Junqi Tang
• Abstract summary: We propose a new paradigm for designing efficient deep unrolling networks using operator sketching. Our numerical experiments on X-ray CT image reconstruction demonstrate the effectiveness of sketched unrolling schemes.
• Score: 5.025654873456756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we propose a new paradigm for designing efficient deep unrolling networks using operator sketching. The deep unrolling networks are currently the state-of-the-art solutions for imaging inverse problems. However, for high-dimensional imaging tasks, especially the 3D cone-beam X-ray CT and 4D MRI imaging, the deep unrolling schemes typically become inefficient both in terms of memory and computation, due to the need of computing multiple times the high-dimensional forward and adjoint operators. Recently researchers have found that such limitations can be partially addressed by stochastic unrolling with subsets of operators, inspired by the success of stochastic first-order optimization. In this work, we propose a further acceleration upon stochastic unrolling, using sketching techniques to approximate products in the high-dimensional image space. The operator sketching can be jointly applied with stochastic unrolling for the best acceleration and compression performance. Our numerical experiments on X-ray CT image reconstruction demonstrate the remarkable effectiveness of our sketched unrolling schemes.

Related papers

• SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation [57.47730473674261]
  We introduce SwiftSketch, a model for image-conditioned vector sketch generation that can produce high-quality sketches in less than a second. SwiftSketch operates by progressively denoising stroke control points sampled from a Gaussian distribution. ControlSketch is a method that enhances SDS-based techniques by incorporating precise spatial control through a depth-aware ControlNet.
  arXiv  Detail & Related papers  (2025-02-12T18:57:12Z)
• Inter-slice Super-resolution of Magnetic Resonance Images by Pre-training and Self-supervised Fine-tuning [49.197385954021456]
  In clinical practice, 2D magnetic resonance (MR) sequences are widely adopted. While individual 2D slices can be stacked to form a 3D volume, the relatively large slice spacing can pose challenges for visualization and subsequent analysis tasks. To reduce slice spacing, deep-learning-based super-resolution techniques are widely investigated. Most current solutions require a substantial number of paired high-resolution and low-resolution images for supervised training, which are typically unavailable in real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-10T02:20:26Z)
• N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
  In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures. We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our method provides faithful approximations of visibility, depth, and appearance attributes.
  arXiv  Detail & Related papers  (2024-05-25T13:54:34Z)
• Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction [17.254539604491303]
  In this paper, we address the problem of few-shot full 3D head reconstruction. We accomplish this by incorporating a probabilistic shape and appearance prior into coordinate-based representations. We extend the H3DS dataset, which now comprises 60 high-resolution 3D full head scans and their corresponding posed images and masks.
  arXiv  Detail & Related papers  (2023-10-12T07:35:30Z)
• Estimating Extreme 3D Image Rotation with Transformer Cross-Attention [13.82735766201496]
  We propose a cross-attention-based approach that utilizes CNN feature maps and a Transformer-Encoder to compute the cross-attention between the activation maps of the image pairs. It is experimentally shown to outperform contemporary state-of-the-art schemes when applied to commonly used image rotation datasets and benchmarks.
  arXiv  Detail & Related papers  (2023-03-05T09:07:26Z)
• Accelerating Deep Unrolling Networks via Dimensionality Reduction [5.73658856166614]
  Deep unrolling networks are currently the state-of-the-art solutions for imaging inverse problems. For high-dimensional imaging tasks, such as X-ray CT and MRI imaging, the deep unrolling schemes typically become inefficient. We propose a new paradigm for designing efficient deep unrolling networks using dimensionality reduction schemes.
  arXiv  Detail & Related papers  (2022-08-31T11:45:21Z)
• GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction [50.248694764703714]
  Unrolled neural networks have recently achieved state-of-the-art accelerated MRI reconstruction. These networks unroll iterative optimization algorithms by alternating between physics-based consistency and neural-network based regularization. We propose Greedy LEarning for Accelerated MRI reconstruction, an efficient training strategy for high-dimensional imaging settings.
  arXiv  Detail & Related papers  (2022-07-18T06:01:29Z)
• Accelerating Plug-and-Play Image Reconstruction via Multi-Stage Sketched Gradients [5.025654873456756]
  We propose a new paradigm for designing fast plug-and-play (lunch) algorithms using dimensionality reduction techniques. Unlike existing approaches which utilize gradient iterations for acceleration, we propose novel multi-stage sketched gradient iterations.
  arXiv  Detail & Related papers  (2022-03-14T17:12:09Z)
• Unpaired Image Super-Resolution with Optimal Transport Maps [128.1189695209663]
  Real-world image super-resolution (SR) tasks often do not have paired datasets limiting the application of supervised techniques. We propose an algorithm for unpaired SR which learns an unbiased OT map for the perceptual transport cost. Our algorithm provides nearly state-of-the-art performance on the large-scale unpaired AIM-19 dataset.
  arXiv  Detail & Related papers  (2022-02-02T16:21:20Z)
• Stochastic Primal-Dual Deep Unrolling Networks for Imaging Inverse Problems [3.7819322027528113]
  We present a new type of efficient deep-unrolling networks for solving imaging inverse problems. In our unrolling network, we only use a subset of the forward and adjoint operator. Our numerical results demonstrate the effectiveness of our approach in X-ray CT imaging task.
  arXiv  Detail & Related papers  (2021-10-19T16:46:03Z)
• Riggable 3D Face Reconstruction via In-Network Optimization [58.016067611038046]
  This paper presents a method for riggable 3D face reconstruction from monocular images. It jointly estimates a personalized face rig and per-image parameters including expressions, poses, and illuminations. Experiments demonstrate that our method achieves SOTA reconstruction accuracy, reasonable robustness and generalization ability.
  arXiv  Detail & Related papers  (2021-04-08T03:53:20Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.