Daily Paper Briefing

Abstract Overview

ZipSplat is a feed-forward 3D Gaussian Splatting model that predicts a compact scene representation from multi-view images without tying one Gaussian to each input pixel. It extracts dense multi-view visual tokens, compresses them into scene tokens with k-means clustering, refines them with cross- and self-attention, and decodes each token into a group of Gaussians with unconstrained 3D positions. Because clustering is performed at inference time, the same trained model can trade quality for efficiency by varying the compression ratio instead of retraining for different budgets. To make free 3D placement train reliably, the method combines geometric supervision with photometric, perceptual, and depth losses, along with careful initialization and progressive training schedules.

Novelty

The paper's main novelty is to decouple Gaussian prediction from the 2D pixel grid and instead generate Gaussians from dynamically formed scene tokens. This token-based formulation, combined with inference-time feature-space clustering, gives a single model a continuous Gaussian-budget control knob while allowing Gaussians to be placed freely in 3D rather than along viewing rays.

Results

Across DL3DV and RealEstate10K, ZipSplat reports state-of-the-art pose-free novel view synthesis while using about 6× fewer Gaussians than pixel-aligned methods, outperforming YoNoSplat by up to 2.1 dB PSNR on DL3DV and by 1.2 dB on RealEstate10K. The model also generalizes zero-shot to Mip-NeRF360 and ScanNet++, where it outperforms comparable baselines, and its quality remains comparatively stable as the number of context views increases.

Key Points

1. ZipSplat replaces per-pixel Gaussian prediction with a scene-token pipeline: dense multi-view tokens are clustered, refined with attention, and decoded into Gaussian groups with free 3D positions.
2. Its compression ratio is adjustable at inference time, so one trained model can span the quality-efficiency curve without retraining; the paper shows graceful degradation under stronger compression.
3. Experiments indicate better reconstruction quality and better scaling with more input views than pixel-aligned feed-forward baselines, while using substantially fewer Gaussians.

References

• arXiv: https://arxiv.org/abs/2606.05102v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2606.05102v1
• Hugging Face Papers: https://huggingface.co/papers/2606.05102
• Project: https://veichta.com/zipsplat