Differentiable Voxel-based X-ray Rendering Improves Sparse-View 3D CBCT Reconstruction

• URL: http://arxiv.org/abs/2411.19224v2
• Date: Mon, 02 Dec 2024 02:04:53 GMT
• Title: Differentiable Voxel-based X-ray Rendering Improves Sparse-View 3D CBCT Reconstruction
• Authors: Mohammadhossein Momeni, Vivek Gopalakrishnan, Neel Dey, Polina Golland, Sarah Frisken,
• Abstract summary: We present DiffVox, a self-supervised framework for Cone-Beam Computed Tomography (CBCT) reconstruction.<n>As a result, we reconstruct high-fidelity 3D CBCT volumes from fewer X-rays, potentially reducing ionizing radiation exposure and improving diagnostic utility.
• Score: 4.941613865666241
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present DiffVox, a self-supervised framework for Cone-Beam Computed Tomography (CBCT) reconstruction by directly optimizing a voxelgrid representation using physics-based differentiable X-ray rendering. Further, we investigate how the different implementations of the X-ray image formation model in the renderer affect the quality of 3D reconstruction and novel view synthesis. When combined with our regularized voxel-based learning framework, we find that using an exact implementation of the discrete Beer-Lambert law for X-ray attenuation in the renderer outperforms both widely used iterative CBCT reconstruction algorithms and modern neural field approaches, particularly when given only a few input views. As a result, we reconstruct high-fidelity 3D CBCT volumes from fewer X-rays, potentially reducing ionizing radiation exposure and improving diagnostic utility. Our implementation is available at https://github.com/hossein-momeni/DiffVox.

Related papers

• X-Field: A Physically Grounded Representation for 3D X-ray Reconstruction [25.13707706037451]
  X-ray imaging is indispensable in medical diagnostics, yet its use is tightly regulated due to potential health risks. Recent research focuses on generating novel views from sparse inputs and reconstructing Computed Tomography (CT) volumes. We introduce X-Field, the first 3D representation specifically designed for X-ray imaging.
  arXiv  Detail & Related papers  (2025-03-11T16:31:56Z)
• X-LRM: X-ray Large Reconstruction Model for Extremely Sparse-View Computed Tomography Recovery in One Second [52.11676689269379]
  Sparse-view 3D CT reconstruction aims to recover structures from a limited number of 2D X-ray projections. Existing feedforward methods are constrained by the limited capacity of CNN-based architectures and the scarcity of large-scale training datasets. We propose an X-ray Large Reconstruction Model (X-LRM) for extremely sparse-view (10 views) CT reconstruction.
  arXiv  Detail & Related papers  (2025-03-09T01:39:59Z)
• Fan-Beam CT Reconstruction for Unaligned Sparse-View X-ray Baggage Dataset [0.0]
  We present a calibration and reconstruction method using an unaligned sparse multi-view X-ray baggage dataset. Our approach integrates multi-spectral neural attenuation field reconstruction with Linear pushbroom (LPB) camera model pose optimization.
  arXiv  Detail & Related papers  (2024-12-04T05:16:54Z)
• TomoGRAF: A Robust and Generalizable Reconstruction Network for Single-View Computed Tomography [3.1209855614927275]
  Traditional analytical/iterative CT reconstruction algorithms require hundreds of angular data samplings. We develop a novel TomoGRAF framework incorporating the unique X-ray transportation physics to reconstruct high-quality 3D volumes.
  arXiv  Detail & Related papers  (2024-11-12T20:07:59Z)
• DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays [41.393567374399524]
  We propose DiffuX2CT, which models CT reconstruction from ultra-sparse X-rays as a conditional diffusion process. By doing so, DiffuX2CT achieves structure-controllable reconstruction, which enables 3D structural information to be recovered from 2D X-rays. As an extra contribution, we collect a real-world lumbar CT dataset, called LumbarV, as a new benchmark to verify the clinical significance and performance of CT reconstruction from X-rays.
  arXiv  Detail & Related papers  (2024-07-18T14:20:04Z)
• R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction [53.19869886963333]
  3D Gaussian splatting (3DGS) has shown promising results in rendering image and surface reconstruction. This paper introduces R2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction.
  arXiv  Detail & Related papers  (2024-05-31T08:39:02Z)
• Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans [10.72672892416061]
  Over-dosed X-rays superimpose potential risks to human health to some extent. Data-driven algorithms from volume scans to X-ray images are restricted by the scarcity of paired X-ray and volume data. We propose CT2X-GAN to synthesize the X-ray images in an end-to-end manner using the content and style disentanglement from three different image domains.
  arXiv  Detail & Related papers  (2024-04-18T04:25:56Z)
• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs [65.80187860906115]
  We propose a novel approach to improve NeRF's performance with sparse inputs. We first adopt a voxel-based ray sampling strategy to ensure that the sampled rays intersect with a certain voxel in 3D space. We then randomly sample additional points within the voxel and apply a Transformer to infer the properties of other points on each ray, which are then incorporated into the volume rendering.
  arXiv  Detail & Related papers  (2024-03-25T15:56:17Z)
• Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis [88.86777314004044]
  We propose a 3D Gaussian splatting-based framework, namely X-Gaussian, for X-ray novel view visualization. Experiments show that our X-Gaussian outperforms state-of-the-art methods by 6.5 dB while enjoying less than 15% training time and over 73x inference speed.
  arXiv  Detail & Related papers  (2024-03-07T00:12:08Z)
• Scalable 3D Reconstruction From Single Particle X-Ray Diffraction Images Based on Online Machine Learning [34.12502156343611]
  We introduce X-RAI, an online reconstruction framework that estimates the structure of a 3D macromolecule from large X-ray SPI datasets. We demonstrate that X-RAI achieves state-of-the-art performance for small-scale datasets in simulation and challenging experimental settings.
  arXiv  Detail & Related papers  (2023-12-22T04:41:31Z)
• Structure-Aware Sparse-View X-ray 3D Reconstruction [26.91084106735878]
  We propose a framework, Structure-Aware X-ray Neural Radiodensity Fields (SAX-NeRF) for sparse-view X-ray 3D reconstruction. Linefomer captures internal structures of objects in 3D space by modeling the dependencies within each line segment of an X-ray. Experiments on X3D show that SAX-NeRF surpasses previous NeRF-based methods by 12.56 and 2.49 dB on novel view synthesis and CT reconstruction.
  arXiv  Detail & Related papers  (2023-11-18T03:39:02Z)
• Geometry-Aware Attenuation Learning for Sparse-View CBCT Reconstruction [53.93674177236367]
  Cone Beam Computed Tomography (CBCT) plays a vital role in clinical imaging. Traditional methods typically require hundreds of 2D X-ray projections to reconstruct a high-quality 3D CBCT image. This has led to a growing interest in sparse-view CBCT reconstruction to reduce radiation doses. We introduce a novel geometry-aware encoder-decoder framework to solve this problem.
  arXiv  Detail & Related papers  (2023-03-26T14:38:42Z)
• Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement [78.48648360358193]
  We present a novel framework that generates textured surface meshes from images. Our approach begins by efficiently initializing the geometry and view-dependency appearance with a NeRF. We jointly refine the appearance with geometry and bake it into texture images for real-time rendering.
  arXiv  Detail & Related papers  (2023-03-03T17:14:44Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)
• End-To-End Convolutional Neural Network for 3D Reconstruction of Knee Bones From Bi-Planar X-Ray Images [6.645111950779666]
  We present an end-to-end Convolutional Neural Network (CNN) approach for 3D reconstruction of knee bones directly from two bi-planar X-ray images.
  arXiv  Detail & Related papers  (2020-04-02T08:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.