GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis

• URL: http://arxiv.org/abs/2404.03126v1
• Date: Thu, 4 Apr 2024 00:28:50 GMT
• Title: GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis
• Authors: Emmanouil Nikolakakis, Utkarsh Gupta, Jonathan Vengosh, Justin Bui, Razvan Marinescu,
• Abstract summary: GaSpCT is a novel view synthesis and 3D scene representation method used to generate novel projection views for Computer Tomography (CT) scans. We adapt the Gaussian Splatting framework to enable novel view synthesis in CT based on limited sets of 2D image projections. We evaluate the performance of our model using brain CT scans from the Parkinson's Progression Markers Initiative (PPMI) dataset.
• Score: 0.6990493129893112
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present GaSpCT, a novel view synthesis and 3D scene representation method used to generate novel projection views for Computer Tomography (CT) scans. We adapt the Gaussian Splatting framework to enable novel view synthesis in CT based on limited sets of 2D image projections and without the need for Structure from Motion (SfM) methodologies. Therefore, we reduce the total scanning duration and the amount of radiation dose the patient receives during the scan. We adapted the loss function to our use-case by encouraging a stronger background and foreground distinction using two sparsity promoting regularizers: a beta loss and a total variation (TV) loss. Finally, we initialize the Gaussian locations across the 3D space using a uniform prior distribution of where the brain's positioning would be expected to be within the field of view. We evaluate the performance of our model using brain CT scans from the Parkinson's Progression Markers Initiative (PPMI) dataset and demonstrate that the rendered novel views closely match the original projection views of the simulated scan, and have better performance than other implicit 3D scene representations methodologies. Furthermore, we empirically observe reduced training time compared to neural network based image synthesis for sparse-view CT image reconstruction. Finally, the memory requirements of the Gaussian Splatting representations are reduced by 17% compared to the equivalent voxel grid image representations.

Related papers

• FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training [15.634646420318731]
  We present a 3D Gaussian-based novel view synthesis method using sparse input images. We propose a multi-stage training scheme with matching-based consistency constraints imposed on the novel views. This is achieved by using the matches of the available training images to supervise the generation of the novel views.
  arXiv  Detail & Related papers  (2024-11-04T16:21:00Z)
• No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images [100.80376573969045]
  NoPoSplat is a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from multi-view images. Our model achieves real-time 3D Gaussian reconstruction during inference. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-31T17:58:22Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Learning 3D Gaussians for Extremely Sparse-View Cone-Beam CT Reconstruction [9.848266253196307]
  Cone-Beam Computed Tomography (CBCT) is an indispensable technique in medical imaging, yet the associated radiation exposure raises concerns in clinical practice. We propose a novel reconstruction framework, namely DIF-Gaussian, which leverages 3D Gaussians to represent the feature distribution in the 3D space. We evaluate DIF-Gaussian on two public datasets, showing significantly superior reconstruction performance than previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-01T08:48:04Z)
• CoCPF: Coordinate-based Continuous Projection Field for Ill-Posed Inverse Problem in Imaging [78.734927709231]
  Sparse-view computed tomography (SVCT) reconstruction aims to acquire CT images based on sparsely-sampled measurements. Due to ill-posedness, implicit neural representation (INR) techniques may leave considerable holes'' (i.e., unmodeled spaces) in their fields, leading to sub-optimal results. We propose the Coordinate-based Continuous Projection Field (CoCPF), which aims to build hole-free representation fields for SVCT reconstruction.
  arXiv  Detail & Related papers  (2024-06-21T08:38:30Z)
• R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction [53.19869886963333]
  3D Gaussian splatting (3DGS) has shown promising results in rendering image and surface reconstruction. This paper introduces R2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction.
  arXiv  Detail & Related papers  (2024-05-31T08:39:02Z)
• Sparse-view CT Reconstruction with 3D Gaussian Volumetric Representation [13.667470059238607]
  Sparse-view CT is a promising strategy for reducing the radiation dose of traditional CT scans. Recently, 3D Gaussian has been applied to model complex natural scenes. We investigate their potential for sparse-view CT reconstruction.
  arXiv  Detail & Related papers  (2023-12-25T09:47:33Z)
• pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction [26.72289913260324]
  pixelSplat is a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images. Our model features real-time and memory-efficient rendering for scalable training as well as fast 3D reconstruction at inference time.
  arXiv  Detail & Related papers  (2023-12-19T17:03:50Z)
• GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis [70.24111297192057]
  We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting.
  arXiv  Detail & Related papers  (2023-12-04T18:59:55Z)
• Generative Modeling in Sinogram Domain for Sparse-view CT Reconstruction [12.932897771104825]
  radiation dose in computed tomography (CT) examinations can be significantly reduced by intuitively decreasing the number of projection views. Previous deep learning techniques with sparse-view data require sparse-view/full-view CT image pairs to train the network with supervised manners. We present a fully unsupervised score-based generative model in sinogram domain for sparse-view CT reconstruction.
  arXiv  Detail & Related papers  (2022-11-25T06:49:18Z)
• Enhancement of Novel View Synthesis Using Omnidirectional Image Completion [61.78187618370681]
  We present a method for synthesizing novel views from a single 360-degree RGB-D image based on the neural radiance field (NeRF) Experiments demonstrated that the proposed method can synthesize plausible novel views while preserving the features of the scene for both artificial and real-world data.
  arXiv  Detail & Related papers  (2022-03-18T13:49:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.