NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation

• URL: http://arxiv.org/abs/2405.14241v1
• Date: Thu, 23 May 2024 07:21:01 GMT
• Title: NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation
• Authors: Chaokang Jiang, Dalong Du, Jiuming Liu, Siting Zhu, Zhenqiang Liu, Zhuang Ma, Zhujin Liang, Jie Zhou,
• Abstract summary: Interpolation confronts challenges from point sparsity, complextemporal dynamics, and the difficulty of deriving complete 3D point clouds from sparse temporal information. This paper presents NeuroGauss4D-corrupt, which excels at modeling complex non-rigid deformations across varied dynamic scenes. NeuroGauss4D-corrupt delivers leading performance on both object-level tasks and large-scale autonomous driving datasets.
• Score: 19.28734823769732
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Point Cloud Interpolation confronts challenges from point sparsity, complex spatiotemporal dynamics, and the difficulty of deriving complete 3D point clouds from sparse temporal information. This paper presents NeuroGauss4D-PCI, which excels at modeling complex non-rigid deformations across varied dynamic scenes. The method begins with an iterative Gaussian cloud soft clustering module, offering structured temporal point cloud representations. The proposed temporal radial basis function Gaussian residual utilizes Gaussian parameter interpolation over time, enabling smooth parameter transitions and capturing temporal residuals of Gaussian distributions. Additionally, a 4D Gaussian deformation field tracks the evolution of these parameters, creating continuous spatiotemporal deformation fields. A 4D neural field transforms low-dimensional spatiotemporal coordinates ($x,y,z,t$) into a high-dimensional latent space. Finally, we adaptively and efficiently fuse the latent features from neural fields and the geometric features from Gaussian deformation fields. NeuroGauss4D-PCI outperforms existing methods in point cloud frame interpolation, delivering leading performance on both object-level (DHB) and large-scale autonomous driving datasets (NL-Drive), with scalability to auto-labeling and point cloud densification tasks. The source code is released at https://github.com/jiangchaokang/NeuroGauss4D-PCI.

Related papers

• Speedy Deformable 3D Gaussian Splatting: Fast Rendering and Compression of Dynamic Scenes [57.69608119350651]
  Recent extensions of 3D Gaussian Splatting (3DGS) to dynamic scenes achieve high-quality novel view synthesis by using neural networks to predict the time-varying deformation of each Gaussian.<n>However, performing per-Gaussian neural inference at every frame poses a significant bottleneck, limiting rendering speed and increasing memory and compute requirements.<n>We present Speedy Deformable 3D Gaussian Splatting (SpeeDe3DGS), a general pipeline for accelerating the rendering speed of dynamic 3DGS and 4DGS representations by reducing neural inference through two complementary techniques.
  arXiv  Detail & Related papers  (2025-06-09T16:30:48Z)
• Learning Bijective Surface Parameterization for Inferring Signed Distance Functions from Sparse Point Clouds with Grid Deformation [50.26314343851213]
  Inferring signed distance functions (SDFs) from sparse point clouds remains a challenge in surface reconstruction. We present a novel approach that learns a dynamic deformation network to predict SDFs in an end-to-end manner. Experimental results on synthetic and real scanned datasets demonstrate that our method significantly outperforms the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-31T02:27:02Z)
• 4Deform: Neural Surface Deformation for Robust Shape Interpolation [47.47045870313048]
  We develop a new approach to generate realistic intermediate shapes between non-rigidly deformed shapes in unstructured data. Our method learns a continuous velocity field in Euclidean space and does not require intermediate-shape supervision during training. For the first time, our method enables new applications like 4D Kinect sequence upsampling and real-world high-resolution mesh deformation.
  arXiv  Detail & Related papers  (2025-02-27T15:47:49Z)
• Point Cloud Denoising With Fine-Granularity Dynamic Graph Convolutional Networks [58.050130177241186]
  Noise perturbations often corrupt 3-D point clouds, hindering downstream tasks such as surface reconstruction, rendering, and further processing. This paper introduces finegranularity dynamic graph convolutional networks called GDGCN, a novel approach to denoising in 3-D point clouds.
  arXiv  Detail & Related papers  (2024-11-21T14:19:32Z)
• GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction [3.043712258792239]
  We present a unified framework integrating neural SDF with 3DGS. This framework incorporates a learnable neural SDF field to guide the densification and pruning of Gaussians. Our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-05-30T03:46:59Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• Mesh-based Gaussian Splatting for Real-time Large-scale Deformation [58.18290393082119]
  It is challenging for users to directly deform or manipulate implicit representations with large deformations in the real-time fashion. We develop a novel GS-based method that enables interactive deformation. Our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate.
  arXiv  Detail & Related papers  (2024-02-07T12:36:54Z)
• GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis [17.572987038801475]
  We propose a method for dynamic scene reconstruction using deformable 3D Gaussians. The differentiable pipeline is optimized end-to-end with a self-supervised rendering. Our method results are comparable to state-of-the-art neural radiance field methods.
  arXiv  Detail & Related papers  (2023-12-18T18:59:03Z)
• Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle [9.082693946898733]
  We introduce a novel point-based approach for fast dynamic scene reconstruction and real-time rendering from both multi-view and monocular videos. In contrast to the prevalent NeRF-based approaches hampered by slow training and rendering speeds, our approach harnesses recent advancements in point-based 3D Gaussian Splatting (3DGS) Our proposed approach showcases a substantial efficiency improvement, achieving a $5times$ faster training speed compared to the per-frame 3DGS modeling.
  arXiv  Detail & Related papers  (2023-12-06T11:25:52Z)
• Exploring Geometric Deep Learning For Precipitation Nowcasting [28.44612565923532]
  We propose a geometric deep learning-based temporal Graph Convolutional Network (GCN) for precipitation nowcasting. The adjacency matrix that simulates the interactions among grid cells is learned automatically by minimizing the L1 loss between prediction and ground truth pixel value. We test the model on sequences of radar reflectivity maps over the Trento/Italy area.
  arXiv  Detail & Related papers  (2023-09-11T21:14:55Z)
• Learning Spatial and Temporal Variations for 4D Point Cloud Segmentation [0.39373541926236766]
  We argue that the temporal information across the frames provides crucial knowledge for 3D scene perceptions. We design a temporal variation-aware module and a temporal voxel-point refiner to capture the temporal variation in the 4D point cloud.
  arXiv  Detail & Related papers  (2022-07-11T07:36:26Z)
• 4DAC: Learning Attribute Compression for Dynamic Point Clouds [37.447460254690135]
  We study the attribute (e.g., color) compression of dynamic point clouds and present a learning-based framework, termed 4DAC. To reduce temporal redundancy within data, we first build the 3D motion estimation and motion compensation modules with deep neural networks. In addition, we also propose a deep conditional entropy model to estimate the probability distribution of the transformed coefficients.
  arXiv  Detail & Related papers  (2022-04-25T15:30:06Z)
• Learning Smooth Neural Functions via Lipschitz Regularization [92.42667575719048]
  We introduce a novel regularization designed to encourage smooth latent spaces in neural fields. Compared with prior Lipschitz regularized networks, ours is computationally fast and can be implemented in four lines of code.
  arXiv  Detail & Related papers  (2022-02-16T21:24:54Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)
• PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling [103.09504572409449]
  We propose a novel deep neural network based method, called PUGeo-Net, to generate uniform dense point clouds. Thanks to its geometry-centric nature, PUGeo-Net works well for both CAD models with sharp features and scanned models with rich geometric details.
  arXiv  Detail & Related papers  (2020-02-24T14:13:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.