Inst4DGS: Instance-Decomposed 4D Gaussian Splatting with Multi-Video Label Permutation Learning

• URL: http://arxiv.org/abs/2603.18402v1
• Date: Thu, 19 Mar 2026 01:49:20 GMT
• Title: Inst4DGS: Instance-Decomposed 4D Gaussian Splatting with Multi-Video Label Permutation Learning
• Abstract summary: Inst4DGS is an instance-decomposed 4D Gaussian Splatting (4DGS) approach with long-horizon per-Gaussian trajectories.<n>We introduce per-video label-permutation latents that learn cross-video instance matches through a differentiable Sinkhorn layer.<n>This explicit label alignment yields sharp decision boundaries and temporally stable identities without identity drift.
• Score: 52.867634524554795
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Inst4DGS, an instance-decomposed 4D Gaussian Splatting (4DGS) approach with long-horizon per-Gaussian trajectories. While dynamic 4DGS has advanced rapidly, instance-decomposed 4DGS remains underexplored, largely due to the difficulty of associating inconsistent instance labels across independently segmented multi-view videos. We address this challenge by introducing per-video label-permutation latents that learn cross-video instance matches through a differentiable Sinkhorn layer, enabling direct multi-view supervision with consistent identity preservation. This explicit label alignment yields sharp decision boundaries and temporally stable identities without identity drift. To further improve efficiency, we propose instance-decomposed motion scaffolds that provide low-dimensional motion bases per object for long-horizon trajectory optimization. Experiments on Panoptic Studio and Neural3DV show that Inst4DGS jointly supports tracking and instance decomposition while achieving state-of-the-art rendering and segmentation quality. On the Panoptic Studio dataset, Inst4DGS improves PSNR from 26.10 to 28.36, and instance mIoU from 0.6310 to 0.9129, over the strongest baseline.

Related papers

• Multi4D: High-Fidelity Dynamic Gaussian Splatting via Multi-Level Competitive Allocation [11.135752651274663]
  We introduce Multi4D, a framework for high-fidelity dynamic Gaussian splatting.<n>We distribute modeling capacity across three levels: static structure, persistent dynamic geometry, and transient appearance primitives.<n>This allocation preserves long-term motion consistency while capturing fine dynamic detail, achieving state-of-the-art rendering quality and real-time performance.
  arXiv  Detail & Related papers  (2026-06-20T19:11:26Z)
• NG-GS: NeRF-Guided 3D Gaussian Splatting Segmentation [65.34304674634713]
  We introduce NG-GS, a framework for high-quality object segmentation in 3DGS that explicitly addresses boundary discretization.<n>We show that our method achieves photorealistic state-of-the-art performance, with significant gains in boundary mIoU.
  arXiv  Detail & Related papers  (2026-04-16T07:14:07Z)
• TRiGS: Temporal Rigid-Body Motion for Scalable 4D Gaussian Splatting [40.83830884179225]
  TRiGS is a novel 4D representation that utilizes unified, continuous geometric transformations.<n>We show that TRiGS achieves high fidelity rendering on standard benchmarks while uniquely scaling to extended video sequences without severe memory bottlenecks.
  arXiv  Detail & Related papers  (2026-04-01T06:35:13Z)
• Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation [76.21162972133534]
  We represent a decomposed 4D scene with Freetime FeatureGS.<n>We design a streaming feature learning strategy to accurately recover it from per-image segmentation maps.<n> Experimental results on several datasets show that the reconstruction quality of our method outperforms recent methods by a large margin.
  arXiv  Detail & Related papers  (2025-12-28T02:37:12Z)
• Tracking-Guided 4D Generation: Foundation-Tracker Motion Priors for 3D Model Animation [21.075786141331974]
  We present emphTrack4DGen, a framework for generating dynamic 4D objects from sparse inputs.<n>In Stage One, we enforce dense, feature-level point correspondences inside the diffusion generator.<n>In Stage Two, we reconstruct a dynamic 4D-GS using a hybrid motion encoding.
  arXiv  Detail & Related papers  (2025-12-05T21:13:04Z)
• TED-4DGS: Temporally Activated and Embedding-based Deformation for 4DGS Compression [14.026420167067117]
  We present TED-4DGS, a temporally activated and embedding-based deformation scheme for rate-distortion-optimized 4DGS compression.<n>Our scheme achieves state-of-the-art rate-distortion performance on several real-world datasets.
  arXiv  Detail & Related papers  (2025-12-05T05:46:35Z)
• SyncTrack4D: Cross-Video Motion Alignment and Video Synchronization for Multi-Video 4D Gaussian Splatting [50.69165364520998]
  We present a novel multi-video 4D Gaussian Splatting (4DGS) approach designed to handle real-world, unsynchronized video sets.<n>Our approach, SyncTrack4D, directly leverages dense 4D track representation of dynamic scene parts as cues for simultaneous cross-video synchronization and 4DGS reconstruction.<n>We evaluate our approach on the Panoptic Studio and SyncNeRF Blender, demonstrating sub-frame synchronization accuracy with an average temporal error below 0.26 frames, and high-fidelity 4D reconstruction reaching 26.3 PSNR scores.
  arXiv  Detail & Related papers  (2025-12-03T23:05:01Z)
• Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models [79.06910348413861]
  We introduce Diff4Splat, a feed-forward method that synthesizes controllable and explicit 4D scenes from a single image.<n>Given a single input image, a camera trajectory, and an optional text prompt, Diff4Splat directly predicts a deformable 3D Gaussian field that encodes appearance, geometry, and motion.
  arXiv  Detail & Related papers  (2025-11-01T11:16:25Z)
• Segmentation-Driven Initialization for Sparse-view 3D Gaussian Splatting [0.9251324073335035]
  3D Gaussian Splatting (3DGS) has enabled real-time rendering with competitive quality.<n>Existing pipelines often rely on Structure-from-Motion (SfM) for camera pose estimation, an approach that struggles in genuinely sparse-view settings.<n>We propose a method that mitigates inefficiency by leveraging region-based segmentation to identify and retain only structurally significant regions.
  arXiv  Detail & Related papers  (2025-09-15T12:31:33Z)
• Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video [55.704264233274294]
  We propose Deblur4DGS to reconstruct a high-quality 4D model from blurry monocular video.<n>We transform continuous dynamic representations within an exposure time into the exposure time estimation.<n>Beyond novel-view synthesis, Deblur4DGS can be applied to improve blurry video from multiple perspectives.
  arXiv  Detail & Related papers  (2024-12-09T12:02:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.