Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy

• URL: http://arxiv.org/abs/2601.22026v1
• Date: Thu, 29 Jan 2026 17:33:14 GMT
• Title: Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy
• Authors: Constantin Kleinbeck, Luisa Theelke, Hannah Schieber, Ulrich Eck, Rüdiger von Eisenhart-Rothe, Daniel Roth,
• Abstract summary: Traditional 2D slices provide little support for interpreting spatial relationships.<n>Direct path tracing and VR rendering can improve perception but are computationally expensive.<n>We propose a hybrid rendering approach for high-quality, interactive, and immersive anatomical visualization.
• Score: 6.112599804159643
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Volumetric medical imaging offers great potential for understanding complex pathologies. Yet, traditional 2D slices provide little support for interpreting spatial relationships, forcing users to mentally reconstruct anatomy into three dimensions. Direct volumetric path tracing and VR rendering can improve perception but are computationally expensive, while precomputed representations, like Gaussian Splatting, require planning ahead. Both approaches limit interactive use. We propose a hybrid rendering approach for high-quality, interactive, and immersive anatomical visualization. Our method combines streamed foveated path tracing with a lightweight Gaussian Splatting approximation of the periphery. The peripheral model generation is optimized with volume data and continuously refined using foveal renderings, enabling interactive updates. Depth-guided reprojection further improves robustness to latency and allows users to balance fidelity with refresh rate. We compare our method against direct path tracing and Gaussian Splatting. Our results highlight how their combination can preserve strengths in visual quality while re-generating the peripheral model in under a second, eliminating extensive preprocessing and approximations. This opens new options for interactive medical visualization.

Related papers

• Joint Semantic and Rendering Enhancements in 3D Gaussian Modeling with Anisotropic Local Encoding [86.55824709875598]
  We propose a joint enhancement framework for 3D semantic Gaussian modeling that synergizes both semantic and rendering branches.<n>Unlike conventional point cloud shape encoding, we introduce an anisotropic 3D Gaussian Chebyshev descriptor to capture fine-grained 3D shape details.<n>We employ a cross-scene knowledge transfer module to continuously update learned shape patterns, enabling faster convergence and robust representations.
  arXiv  Detail & Related papers  (2026-01-05T18:33:50Z)
• Fast and Explicit: Slice-to-Volume Reconstruction via 3D Gaussian Primitives with Analytic Point Spread Function Modeling [16.058575857567373]
  High-resolution 3D reconstruction of the brain from motion-corrupted low-resolution 2D acquisitions is a prerequisite for accurate neurodevelopmental diagnosis.<n>In this work, we propose a shift from neural network based representations to explicit representations.<n>Our framework paves the way towards translation into clinical routine of real-time fetal 3D MRI.
  arXiv  Detail & Related papers  (2025-12-12T15:01:30Z)
• Generative AI Pipeline for Interactive Prompt-driven 2D-to-3D Vascular Reconstruction for Fontan Geometries from Contrast-Enhanced X-Ray Fluoroscopy Imaging [0.0]
  Current assessment of Fontan palliation relies on fluoroscopic angiography.<n>A multi-step AI pipeline was developed for systematic, iterative processing of fluoroscopic angiograms.<n>The pipeline successfully generated geometrically optimized 2D projections from single-view angiograms.
  arXiv  Detail & Related papers  (2025-09-16T04:47:25Z)
• ClipGS: Clippable Gaussian Splatting for Interactive Cinematic Visualization of Volumetric Medical Data [51.095474325541794]
  We introduce ClipGS, an innovative Gaussian splatting framework with the clipping plane supported, for interactive cinematic visualization of medical data.<n>We validate our method on five volumetric medical data, and reach an average 36.635 PSNR rendering quality with 156 FPS and 16.1 MB model size.
  arXiv  Detail & Related papers  (2025-07-09T08:24:28Z)
• JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splatting [10.690965024885358]
  Reconstructing 3D scenes from sparse viewpoints is a long-standing challenge with wide applications.<n>Recent advances in feed-forward 3D Gaussian sparse-view reconstruction methods provide an efficient solution for real-time novel view synthesis.<n>We propose JointSplat, a unified framework that leverages the complementarity between optical flow and depth.
  arXiv  Detail & Related papers  (2025-06-04T12:04:40Z)
• LinPrim: Linear Primitives for Differentiable Volumetric Rendering [51.56484100374058]
  We introduce two new scene representations based on linear primitives.<n>We present a different octaiableizer that runs efficiently on GPU.<n>We demonstrate comparable performance to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-27T18:49:38Z)
• GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views [67.34073368933814]
  We propose a generalizable Gaussian Splatting approach for high-resolution image rendering under a sparse-view camera setting. We train our Gaussian parameter regression module on human-only data or human-scene data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. Experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.
  arXiv  Detail & Related papers  (2024-11-18T08:18:44Z)
• Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization [1.0580610673031074]
  In medical image visualization, path tracing of volumetric medical data like CT scans produces lifelike visualizations. We propose a novel approach utilizing GS to create an efficient but static intermediate representation of CT scans. Our approach achieves interactive frame rates while preserving anatomical structures, with quality adjustable to the target hardware.
  arXiv  Detail & Related papers  (2024-10-22T12:56:58Z)
• GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis [70.24111297192057]
  We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting.
  arXiv  Detail & Related papers  (2023-12-04T18:59:55Z)
• Self-Supervised Generative-Contrastive Learning of Multi-Modal Euclidean Input for 3D Shape Latent Representations: A Dynamic Switching Approach [53.376029341079054]
  We propose a combined generative and contrastive neural architecture for learning latent representations of 3D shapes.<n>The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape.
  arXiv  Detail & Related papers  (2023-01-11T18:14:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.