CAGS: Color-Adaptive Volumetric Video Streaming with Dynamic 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2605.09279v1
• Date: Sun, 10 May 2026 03:08:24 GMT
• Title: CAGS: Color-Adaptive Volumetric Video Streaming with Dynamic 3D Gaussian Splatting
• Abstract summary: Volumetric video (VV) streaming enables real-time, immersive access to remote 3D environments, powering telepresence, ecological monitoring, and robotic teleoperation.<n>These applications turn VV streaming into a real-time interface to remote physical environments, imposing new system-level demands for photorealistic scene representation, low-latency interaction, and robust performance under heterogeneous networks.<n>3D Gaussian Splatting (3DGS) has been widely used for real-time rendering, offering superior visual quality and rendering performance, but it faces challenges due to bandwidth consumption.<n>We propose a novel Color-Adaptive scheme for adaptive VV streaming that uses vector quant
• Score: 24.443567258025066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Volumetric video (VV) streaming enables real-time, immersive access to remote 3D environments, powering telepresence, ecological monitoring, and robotic teleoperation. These applications turn VV streaming into a real-time interface to remote physical environments, imposing new system-level demands for photorealistic scene representation, low-latency interaction, and robust performance under heterogeneous networks. 3D Gaussian Splatting (3DGS) has been widely used for real-time photorealistic rendering, offering superior visual quality and rendering performance, but it faces challenges due to bandwidth consumption. Furthermore, as the foundation of adaptive VV streaming, existing Levels of Detail (LoD) methods based on density are not well-suited to Gaussian representations, leading to visible gaps and severe quality degradation. Recent studies have also explored attribute compression techniques to reduce bandwidth consumption. Our preliminary studies reveal that aggressive attribute compression primarily causes color distortion, which can be effectively corrected in the rendered image using a reference image. Motivated by these findings, we propose a novel Color-Adaptive scheme for adaptive VV streaming that uses vector quantization (VQ) to establish LoDs and correct color distortions with low-resolution reference images. We further present CAGS, an adaptive VV streaming system compatible with diverse Gaussian representations, which integrates the Color-Adaptive scheme by rendering reference images on the streaming server and performing color restoration on the client. Extensive experiments on our prototype system demonstrate that CAGS outperforms the existing adaptive streaming systems in PSNR by 5$\sim$20 dB under fluctuating bandwidth, operates significantly faster than existing scalable Gaussian compression methods, and generalizes across different Gaussian representations.

Related papers

• Seeing What Matters: Perceptual Wrapper with Common Randomness for 3D Gaussian Splatting [8.053023331297602]
  3D Gaussian Splatting (3DGS) achieves real-time rendering but struggles to synthesize high-frequency textures.<n>We propose a versatile 2D perceptual wrapper that enhances the rendered outputs of existing 3DGS representations in a content- and view-dependent manner.
  arXiv  Detail & Related papers  (2026-06-10T08:14:47Z)
• RAVE: Rate-Adaptive Visual Encoding for 3D Gaussian Splatting [17.19039932786604]
  We propose a flexible compression scheme for 3DGS that supports at any rate between predefined bounds.<n>Our method is computationally lightweight, requires no retraining for any rate, and preserves rendering quality across a broad range of operating points.<n> Experiments demonstrate that the approach achieves efficient, high-quality compression while offering dynamic rate control, making it suitable for practical deployment in immersive applications.
  arXiv  Detail & Related papers  (2025-12-07T23:59:46Z)
• GFix: Perceptually Enhanced Gaussian Splatting Video Compression [16.637492219941446]
  Existing 3DGS-based video codecs exhibit more noticeable visual artifacts and relatively low compression ratios.<n>In this paper, we specifically target the perceptual enhancement of 3DGS-based video compression, based on the assumption that artifacts from 3DGS rendering and quantization resemble noisy latents sampled during diffusion training.<n>We propose a content-adaptive framework, GFix, comprising a streamlined, single-step diffusion model that serves as an off-the-shelf neural enhancer.
  arXiv  Detail & Related papers  (2025-11-10T11:03:30Z)
• 4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming [52.76837132019501]
  We introduce 4DGCPro, a novel hierarchical 4D compression framework.<n>4DGCPro facilitates real-time mobile decoding and high-quality rendering via progressive volumetric video streaming.<n>We present an end-to-end entropy-optimized training scheme.
  arXiv  Detail & Related papers  (2025-09-22T08:38:17Z)
• Adaptive 3D Gaussian Splatting Video Streaming [28.283254336752602]
  We introduce an innovative framework for 3DGS volumetric video streaming.<n>By employing hybrid saliency tiling and differentiated quality modeling, we achieve efficient data compression and adaptation to bandwidth fluctuations.<n>Our method demonstrated superiority over existing approaches in various aspects, including video quality, compression effectiveness, and transmission rate.
  arXiv  Detail & Related papers  (2025-07-19T01:45:24Z)
• 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video [56.04182926886754]
  3D Gaussian Splatting (3DGS) has substantial potential for enabling photorealistic Free-Viewpoint Video (FVV) experiences.<n>Existing methods typically handle dynamic 3DGS representation and compression separately, motion information and the rate-distortion trade-off during training.<n>We propose 4DGC, a rate-aware 4D Gaussian compression framework that significantly reduces storage size while maintaining superior RD performance for FVV.
  arXiv  Detail & Related papers  (2025-03-24T08:05:27Z)
• BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Video [58.97416204208624]
  We present BEAM, a novel pipeline that bridges 4D Gaussian representations with physically-based rendering (PBR) to produce high-quality, relightable videos.<n>By offering realistic, lifelike visualizations under diverse lighting conditions, BEAM opens new possibilities for interactive entertainment, storytelling, and creative visualization.
  arXiv  Detail & Related papers  (2025-02-12T10:58:09Z)
• VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression [59.14355576912495]
  NeRF-based video has revolutionized visual media by delivering photorealistic Free-Viewpoint Video (FVV) experiences.<n>The substantial data volumes pose significant challenges for storage and transmission.<n>We propose VRVVC, a novel end-to-end joint variable-rate framework for video compression.
  arXiv  Detail & Related papers  (2024-12-16T01:28:04Z)
• LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming [4.209963145038135]
  XR requires efficient streaming of 3D online worlds, challenging current 3DGS representations to adapt to bandwidth-constrained environments.<n>This paper proposes LapisGS, a layered 3DGS that supports adaptive streaming and progressive rendering.
  arXiv  Detail & Related papers  (2024-08-27T07:06:49Z)
• Image-GS: Content-Adaptive Image Representation via 2D Gaussians [52.598772767324036]
  We introduce Image-GS, a content-adaptive image representation based on 2D Gaussians radiance.<n>It supports hardware-friendly rapid access for real-time usage, requiring only 0.3K MACs to decode a pixel.<n>We demonstrate its versatility with several applications, including texture compression, semantics-aware compression, and joint image compression and restoration.
  arXiv  Detail & Related papers  (2024-07-02T00:45:21Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.