Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging

• URL: http://arxiv.org/abs/2208.12737v1
• Date: Fri, 26 Aug 2022 15:49:28 GMT
• Title: Fast Auto-Differentiable Digitally Reconstructed Radiographs for Solving Inverse Problems in Intraoperative Imaging
• Authors: Vivek Gopalakrishnan and Polina Golland
• Abstract summary: Digitally reconstructed radiographs (DRRs) are well-studied in preoperative settings. DRRs can be used to solve inverse problems such as slice-to-volume registration and 3D reconstruction.
• Score: 2.6027967363792865
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The use of digitally reconstructed radiographs (DRRs) to solve inverse problems such as slice-to-volume registration and 3D reconstruction is well-studied in preoperative settings. In intraoperative imaging, the utility of DRRs is limited by the challenges in generating them in real-time and supporting optimization procedures that rely on repeated DRR synthesis. While immense progress has been made in accelerating the generation of DRRs through algorithmic refinements and GPU implementations, DRR-based optimization remains slow because most DRR generators do not offer a straightforward way to obtain gradients with respect to the imaging parameters. To make DRRs interoperable with gradient-based optimization and deep learning frameworks, we have reformulated Siddon's method, the most popular ray-tracing algorithm used in DRR generation, as a series of vectorized tensor operations. We implemented this vectorized version of Siddon's method in PyTorch, taking advantage of the library's strong automatic differentiation engine to make this DRR generator fully differentiable with respect to its parameters. Additionally, using GPU-accelerated tensor computation enables our vectorized implementation to achieve rendering speeds equivalent to state-of-the-art DRR generators implemented in CUDA and C++. We illustrate the resulting method in the context of slice-to-volume registration. Moreover, our simulations suggest that the loss landscapes for the slice-to-volume registration problem are convex in the neighborhood of the optimal solution, and gradient-based registration promises a much faster solution than prevailing gradient-free optimization strategies. The proposed DRR generator enables fast computer vision algorithms to support image guidance in minimally invasive procedures. Our implementation is publically available at https://github.com/v715/DiffDRR.

Related papers

• Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging [45.39911367007956]
  Deep-unrolling and plug-and-play approaches have become the de-facto for single-pixel imaging (SPI) inverse problem.<n>In this paper, we address the challenge of integrating the strengths of both classes of solvers.
  arXiv  Detail & Related papers  (2025-05-29T07:16:57Z)
• Pixel to Gaussian: Ultra-Fast Continuous Super-Resolution with 2D Gaussian Modeling [50.34513854725803]
  Arbitrary-scale super-resolution (ASSR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs with arbitrary upsampling factors. We propose a novel ContinuousSR framework with a Pixel-to-Gaussian paradigm, which explicitly reconstructs 2D continuous HR signals from LR images using Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-03-09T13:43:57Z)
• vSHARP: variable Splitting Half-quadratic Admm algorithm for Reconstruction of inverse-Problems [7.043932618116216]
  vSHARP (variable Splitting Half-quadratic ADMM algorithm for Reconstruction of inverse Problems) is a novel Deep Learning (DL)-based method for solving ill-posed inverse problems arising in Medical Imaging (MI) For data consistency, vSHARP unrolls a differentiable gradient descent process in the image domain, while a DL-based denoiser, such as a U-Net architecture, is applied to enhance image quality. Our comparative analysis with state-of-the-art methods demonstrates the superior performance of vSHARP in these applications.
  arXiv  Detail & Related papers  (2023-09-18T17:26:22Z)
• Curvature regularization for Non-line-of-sight Imaging from Under-sampled Data [5.591221518341613]
  Non-line-of-sight (NLOS) imaging aims to reconstruct the three-dimensional hidden scenes from the data measured in the line-of-sight. We propose novel NLOS reconstruction models based on curvature regularization. We evaluate the proposed algorithms on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2023-01-01T14:10:43Z)
• DCS-RISR: Dynamic Channel Splitting for Efficient Real-world Image Super-Resolution [15.694407977871341]
  Real-world image super-resolution (RISR) has received increased focus for improving the quality of SR images under unknown complex degradation. Existing methods rely on the heavy SR models to enhance low-resolution (LR) images of different degradation levels. We propose a novel Dynamic Channel Splitting scheme for efficient Real-world Image Super-Resolution, termed DCS-RISR.
  arXiv  Detail & Related papers  (2022-12-15T04:34:57Z)
• RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging [62.315673415889314]
  This paper proposes a deep recurrent Rotation Averaging Graph (RAGO) for Multiple Rotation Averaging (MRA) Our framework is a real-time learning-to-optimize rotation averaging graph with a tiny size deployed for real-world applications.
  arXiv  Detail & Related papers  (2022-12-14T13:19:40Z)
• Loop Unrolled Shallow Equilibrium Regularizer (LUSER) -- A Memory-Efficient Inverse Problem Solver [26.87738024952936]
  In inverse problems we aim to reconstruct some underlying signal of interest from potentially corrupted and often ill-posed measurements. We propose an LU algorithm with shallow equilibrium regularizers (L) These implicit models are as expressive as deeper convolutional networks, but far more memory efficient during training.
  arXiv  Detail & Related papers  (2022-10-10T19:50:37Z)
• Practical Operator Sketching Framework for Accelerating Iterative Data-Driven Solutions in Inverse Problems [13.337452518482717]
  We propose a new operator-sketching paradigm for efficient iterative data-driven reconstruction schemes. These IDR schemes are currently the state-of-the-art solutions for imaging inverse problems.
  arXiv  Detail & Related papers  (2022-08-31T11:45:21Z)
• Nesterov Accelerated ADMM for Fast Diffeomorphic Image Registration [63.15453821022452]
  Recent developments in approaches based on deep learning have achieved sub-second runtimes for DiffIR. We propose a simple iterative scheme that functionally composes intermediate non-stationary velocity fields. We then propose a convex optimisation model that uses a regularisation term of arbitrary order to impose smoothness on these velocity fields.
  arXiv  Detail & Related papers  (2021-09-26T19:56:45Z)
• LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond [75.37541439447314]
  Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. This paper proposes a linearly-assembled pixel-adaptive regression network (LAPAR) to strike a sweet spot of deep model complexity and resulting SISR quality.
  arXiv  Detail & Related papers  (2021-05-21T15:47:18Z)
• Rotation Coordinate Descent for Fast Globally Optimal Rotation Averaging [47.3713707521106]
  We present a fast algorithm that achieves global optimality called rotation coordinate descent (RCD) Unlike block coordinate descent (BCD) which solves SDP by updating the semidefinite matrix in a row-by-row fashion, RCD directly maintains and updates all valid rotations throughout the iterations. We mathematically prove the convergence of our algorithm and empirically show its superior efficiency over state-of-the-art global methods.
  arXiv  Detail & Related papers  (2021-03-15T11:31:34Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• DynaVSR: Dynamic Adaptive Blind Video Super-Resolution [60.154204107453914]
  DynaVSR is a novel meta-learning-based framework for real-world video SR. We train a multi-frame downscaling module with various types of synthetic blur kernels, which is seamlessly combined with a video SR network for input-aware adaptation. Experimental results show that DynaVSR consistently improves the performance of the state-of-the-art video SR models by a large margin.
  arXiv  Detail & Related papers  (2020-11-09T15:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.