GATS: Gaussian Aware Temporal Scaling Transformer for Invariant 4D Spatio-Temporal Point Cloud Representation

• URL: http://arxiv.org/abs/2603.16154v1
• Date: Tue, 17 Mar 2026 06:18:27 GMT
• Title: GATS: Gaussian Aware Temporal Scaling Transformer for Invariant 4D Spatio-Temporal Point Cloud Representation
• Abstract summary: We propose a novel dual invariant framework, termed textbfGaussian Aware Temporal Scaling (GATS), which explicitly resolves both distributional inconsistencies and temporal.<n>Our experiments on mainstream benchmarks MSR-Action3D (textbf+6.62% accuracy), NTU RGBD (textbf+1.4% accuracy), and Synthia4D (textbf+1.8% mIoU) demonstrate significant performance gains.
• Score: 4.686110187936828
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Understanding 4D point cloud videos is essential for enabling intelligent agents to perceive dynamic environments. However, temporal scale bias across varying frame rates and distributional uncertainty in irregular point clouds make it highly challenging to design a unified and robust 4D backbone. Existing CNN or Transformer based methods are constrained either by limited receptive fields or by quadratic computational complexity, while neglecting these implicit distortions. To address this problem, we propose a novel dual invariant framework, termed \textbf{Gaussian Aware Temporal Scaling (GATS)}, which explicitly resolves both distributional inconsistencies and temporal. The proposed \emph{Uncertainty Guided Gaussian Convolution (UGGC)} incorporates local Gaussian statistics and uncertainty aware gating into point convolution, thereby achieving robust neighborhood aggregation under density variation, noise, and occlusion. In parallel, the \emph{Temporal Scaling Attention (TSA)} introduces a learnable scaling factor to normalize temporal distances, ensuring frame partition invariance and consistent velocity estimation across different frame rates. These two modules are complementary: temporal scaling normalizes time intervals prior to Gaussian estimation, while Gaussian modeling enhances robustness to irregular distributions. Our experiments on mainstream benchmarks MSR-Action3D (\textbf{+6.62\%} accuracy), NTU RGBD (\textbf{+1.4\%} accuracy), and Synthia4D (\textbf{+1.8\%} mIoU) demonstrate significant performance gains, offering a more efficient and principled paradigm for invariant 4D point cloud video understanding with superior accuracy, robustness, and scalability compared to Transformer based counterparts.

Related papers

• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis [94.20462261657525]
  GrainGS is a dynamic framework that combines a hierarchical anchor scaffold with per-Gaussian deformation.<n>It achieves high reconstruction quality, real-time novel view synthesis, and compact storage.
  arXiv  Detail & Related papers  (2026-07-23T15:52:35Z)
• Deformable Gaussian Occupancy: Decoupling Rigid and Nonrigid Motion with Factorized Distillation [81.44955493339835]
  DeGO is a deformable Gaussian occupancy framework that unifies decoupled Gaussian deformation with factorized 4D foundation-model distillation.<n> Experiments on the Occ3D-NuScenes benchmark demonstrate that our method achieves state-of-the-art performance under weak supervision.
  arXiv  Detail & Related papers  (2026-05-27T15:09:55Z)
• Normalized Architectures are Natively 4-Bit [49.13186675123547]
  nGPT, an architecture that constrains weights and hidden representations to the unit hypersphere, is inherently more robust to low-precision arithmetic.<n>We validate this approach on both a 1.2B dense model and hybrid (Mamba-Transformer) MoE models of up to 3B/30B parameters.
  arXiv  Detail & Related papers  (2026-05-07T11:54:07Z)
• Faster 3D Gaussian Splatting Convergence via Structure-Aware Densification [52.517763621139956]
  3D Gaussian Splatting has emerged as a powerful scene representation for real-time novel-view synthesis.<n>Standard adaptive density control relies on screen-space positional gradients.<n>We present a structure-aware densification framework for 3D Gaussian Splatting.
  arXiv  Detail & Related papers  (2026-04-30T15:37:20Z)
• GS-STVSR: Ultra-Efficient Continuous Spatio-Temporal Video Super-Resolution via 2D Gaussian Splatting [71.12303582606019]
  Continuous Video Super-Resolution (C-STVSR) aims to simultaneously enhance the spatial and frame rate of videos by arbitrary scale factors.<n>We propose GS-STVSR, an ultra-efficient C-STVSR framework based on 2D Gaussianting (2D-GS)<n>We show that GS-STVSR achieves state-of-the-art quality across all benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T10:11:05Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentation [6.650237349474389]
  We present an efficient learning-free 4D Gaussian segmentation framework that lifts video segmentation masks to 4D spaces.<n>The first stage is an Iterative Gaussian Instance Tracing (IGIT) at the temporal segment level.<n>The second stage is a frame-wise Gaussian Rendering Range Control (RCC) via suppressing highly uncertain Gaussians near object boundaries.<n>Experiments on HyperNeRF and Neu3D demonstrate that our method produces accurate object Gaussian point clouds with clearer boundaries.
  arXiv  Detail & Related papers  (2026-02-09T11:41:06Z)
• TGSFormer: Scalable Temporal Gaussian Splatting for Embodied Semantic Scene Completion [38.31128137855632]
  Embodied 3D Semantic Scene Completion infers dense geometry and semantics from continuous egocentric observations.<n>Recent depth-guided approach alleviates this issue but remains local, suffering from latency and memory overhead as scale increases.<n>We propose TGSFormer, a scalable Temporal Gaussian Splatting framework for embodied SSC.
  arXiv  Detail & Related papers  (2025-11-29T03:47:14Z)
• DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining [21.816338275013702]
  We propose DeLiVR, an efficient video deraining method that injects Lie-group differential biases directly into attention scores of the network.<n>A rotation-bounded Lie relative bias predicts the in-plane angle of each frame using a compact prediction module.<n>A differential group displacement computes angular differences between frames adjacent to estimate a velocity.<n>This bias combines temporal decay and attention masks to focus on inter-frame relationships while precisely matching the direction of rain streaks.
  arXiv  Detail & Related papers  (2025-09-26T00:29:36Z)
• Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression [97.66080040613726]
  We propose a Bidirectional Feature-aligned Motion Transformation (Bi-FMT) framework that implicitly models motion in the feature space.<n>Bi-FMT aligns features across both past and future frames to produce temporally consistent latent representations.<n>We show Bi-FMT surpasses D-DPCC and AdaDPCC in both compression efficiency and runtime.
  arXiv  Detail & Related papers  (2025-09-18T03:51:06Z)
• FRAM: Frobenius-Regularized Assignment Matching with Mixed-Precision Computing [6.987672546471471]
  A Quadratic Assignment Problem (QAP) seeks to establish node correspondences between two graphs.<n>We develop a theoretically grounded mixed-precision architecture to achieve substantial acceleration.<n>FRAM achieves up to 370X speedup over its CPU-FP64 counterpart, with negligible loss in solution accuracy.
  arXiv  Detail & Related papers  (2025-07-26T07:35:09Z)
• Speedy Deformable 3D Gaussian Splatting: Fast Rendering and Compression of Dynamic Scenes [57.69608119350651]
  Recent extensions of 3D Gaussian Splatting (3DGS) to dynamic scenes achieve high-quality novel view synthesis by using neural networks to predict the time-varying deformation of each Gaussian.<n>However, performing per-Gaussian neural inference at every frame poses a significant bottleneck, limiting rendering speed and increasing memory and compute requirements.<n>We present Speedy Deformable 3D Gaussian Splatting (SpeeDe3DGS), a general pipeline for accelerating the rendering speed of dynamic 3DGS and 4DGS representations by reducing neural inference through two complementary techniques.
  arXiv  Detail & Related papers  (2025-06-09T16:30:48Z)
• NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation [19.28734823769732]
  Interpolation confronts challenges from point sparsity, complextemporal dynamics, and the difficulty of deriving complete 3D point clouds from sparse temporal information. This paper presents NeuroGauss4D-corrupt, which excels at modeling complex non-rigid deformations across varied dynamic scenes. NeuroGauss4D-corrupt delivers leading performance on both object-level tasks and large-scale autonomous driving datasets.
  arXiv  Detail & Related papers  (2024-05-23T07:21:01Z)
• Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking [52.393359791978035]
  Motion2VecSets is a 4D diffusion model for dynamic surface reconstruction from point cloud sequences. We parameterize 4D dynamics with latent sets instead of using global latent codes. For more temporally-coherent object tracking, we synchronously denoise deformation latent sets and exchange information across multiple frames.
  arXiv  Detail & Related papers  (2024-01-12T15:05:08Z)
• Uncertainty-Aware Camera Pose Estimation from Points and Lines [101.03675842534415]
  Perspective-n-Point-and-Line (Pn$PL) aims at fast, accurate and robust camera localizations with respect to a 3D model from 2D-3D feature coordinates.
  arXiv  Detail & Related papers  (2021-07-08T15:19:36Z)
• Displacement-Invariant Cost Computation for Efficient Stereo Matching [122.94051630000934]
  Deep learning methods have dominated stereo matching leaderboards by yielding unprecedented disparity accuracy. But their inference time is typically slow, on the order of seconds for a pair of 540p images. We propose a emphdisplacement-invariant cost module to compute the matching costs without needing a 4D feature volume.
  arXiv  Detail & Related papers  (2020-12-01T23:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.