DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering

• URL: http://arxiv.org/abs/2406.02518v2
• Date: Mon, 09 Dec 2024 15:31:08 GMT
• Title: DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering
• Authors: Zhongpai Gao, Benjamin Planche, Meng Zheng, Xiao Chen, Terrence Chen, Ziyan Wu,
• Abstract summary: Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes. We present a novel approach that marries realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation.
• Score: 30.30749508345767
• License:
• Abstract: Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes, widely used in preoperative settings but limited in intraoperative applications due to computational bottlenecks, especially for accurate but heavy physics-based Monte Carlo methods. While analytical DRR renderers offer greater efficiency, they overlook anisotropic X-ray image formation phenomena, such as Compton scattering. We present a novel approach that marries realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). Our direction-disentangled 3DGS (DDGS) method separates the radiosity contribution into isotropic and direction-dependent components, approximating complex anisotropic interactions without intricate runtime simulations. Additionally, we adapt the 3DGS initialization to account for tomography data properties, enhancing accuracy and efficiency. Our method outperforms state-of-the-art techniques in image accuracy. Furthermore, our DDGS shows promise for intraoperative applications and inverse problems such as pose registration, delivering superior registration accuracy and runtime performance compared to analytical DRR methods.

Related papers

• 4DRGS: 4D Radiative Gaussian Splatting for Efficient 3D Vessel Reconstruction from Sparse-View Dynamic DSA Images [49.170407434313475]
  Existing methods often produce suboptimal results or require excessive computation time. We propose 4D radiative Gaussian splatting (4DRGS) to achieve high-quality reconstruction efficiently. 4DRGS achieves impressive results in 5 minutes training, which is 32x faster than the state-of-the-art method.
  arXiv  Detail & Related papers  (2024-12-17T13:51:56Z)
• DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models [67.50989119438508]
  We introduce DSplats, a novel method that directly denoises multiview images using Gaussian-based Reconstructors to produce realistic 3D assets. Our experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction.
  arXiv  Detail & Related papers  (2024-12-11T07:32:17Z)
• Unsupervised Multi-Parameter Inverse Solving for Reducing Ring Artifacts in 3D X-Ray CBCT [35.73129314731503]
  Ring artifacts are prevalent in 3D cone-beam computed tomography (CBCT) due to non-ideal responses of X-ray detectors. Current state-of-the-art (SOTA) ring artifact reduction (RAR) algorithms rely on extensive paired CT samples for supervised learning. We introduce textbfRiner, an unsupervised method formulating 3D CBCT RAR as a multi- parameter inverse problem.
  arXiv  Detail & Related papers  (2024-12-08T08:22:58Z)
• R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction [53.19869886963333]
  3D Gaussian splatting (3DGS) has shown promising results in rendering image and surface reconstruction. This paper introduces R2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction.
  arXiv  Detail & Related papers  (2024-05-31T08:39:02Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• End-to-End Rate-Distortion Optimized 3D Gaussian Representation [33.20840558425759]
  We formulate the compact 3D Gaussian learning as an end-to-end Rate-Distortion Optimization problem. We introduce dynamic pruning and entropy-constrained vector quantization (ECVQ) that optimize the rate and distortion at the same time. We verify our method on both real and synthetic scenes, showcasing that RDO-Gaussian greatly reduces the size of 3D Gaussian over 40x.
  arXiv  Detail & Related papers  (2024-04-09T14:37:54Z)
• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)
• GaussianPro: 3D Gaussian Splatting with Progressive Propagation [49.918797726059545]
  3DGS relies heavily on the point cloud produced by Structure-from-Motion (SfM) techniques. We propose a novel method that applies a progressive propagation strategy to guide the densification of the 3D Gaussians. Our method significantly surpasses 3DGS on the dataset, exhibiting an improvement of 1.15dB in terms of PSNR.
  arXiv  Detail & Related papers  (2024-02-22T16:00:20Z)
• Plug-and-Play Regularization on Magnitude with Deep Priors for 3D Near-Field MIMO Imaging [0.0]
  Near-field radar imaging systems are used in a wide range of applications such as concealed weapon detection and medical diagnosis. We consider the problem of the three-dimensional (3D) complex-valued reflectivity by enforcing regularization on its magnitude.
  arXiv  Detail & Related papers  (2023-12-26T12:25:09Z)
• Sparse-view CT Reconstruction with 3D Gaussian Volumetric Representation [13.667470059238607]
  Sparse-view CT is a promising strategy for reducing the radiation dose of traditional CT scans. Recently, 3D Gaussian has been applied to model complex natural scenes. We investigate their potential for sparse-view CT reconstruction.
  arXiv  Detail & Related papers  (2023-12-25T09:47:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.