Learnable Infinite Taylor Gaussian for Dynamic View Rendering

• URL: http://arxiv.org/abs/2412.04282v1
• Date: Thu, 05 Dec 2024 16:03:37 GMT
• Title: Learnable Infinite Taylor Gaussian for Dynamic View Rendering
• Authors: Bingbing Hu, Yanyan Li, Rui Xie, Bo Xu, Haoye Dong, Junfeng Yao, Gim Hee Lee,
• Abstract summary: This paper introduces a novel approach based on a learnable Taylor Formula to model the temporal evolution of Gaussians. The proposed method achieves state-of-the-art performance in this domain.
• Score: 55.382017409903305
• License:
• Abstract: Capturing the temporal evolution of Gaussian properties such as position, rotation, and scale is a challenging task due to the vast number of time-varying parameters and the limited photometric data available, which generally results in convergence issues, making it difficult to find an optimal solution. While feeding all inputs into an end-to-end neural network can effectively model complex temporal dynamics, this approach lacks explicit supervision and struggles to generate high-quality transformation fields. On the other hand, using time-conditioned polynomial functions to model Gaussian trajectories and orientations provides a more explicit and interpretable solution, but requires significant handcrafted effort and lacks generalizability across diverse scenes. To overcome these limitations, this paper introduces a novel approach based on a learnable infinite Taylor Formula to model the temporal evolution of Gaussians. This method offers both the flexibility of an implicit network-based approach and the interpretability of explicit polynomial functions, allowing for more robust and generalizable modeling of Gaussian dynamics across various dynamic scenes. Extensive experiments on dynamic novel view rendering tasks are conducted on public datasets, demonstrating that the proposed method achieves state-of-the-art performance in this domain. More information is available on our project page(https://ellisonking.github.io/TaylorGaussian).

Related papers

• Optimal Transport-Based Displacement Interpolation with Data Augmentation for Reduced Order Modeling of Nonlinear Dynamical Systems [0.0]
  We present a novel reduced-order Model (ROM) that exploits optimal transport theory and displacement to enhance the representation of nonlinear dynamics in complex systems. We show improved accuracy and efficiency in predicting complex system behaviors, indicating the potential of this approach for a wide range of applications in computational physics and engineering.
  arXiv  Detail & Related papers  (2024-11-13T16:29:33Z)
• Nonstationary Sparse Spectral Permanental Process [24.10531062895964]
  We propose a novel approach utilizing the sparse spectral representation of nonstationary kernels. This technique relaxes the constraints on kernel types and stationarity, allowing for more flexible modeling. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-04T16:40:56Z)
• Latent variable model for high-dimensional point process with structured missingness [4.451479907610764]
  Longitudinal data are important in numerous fields, such as healthcare, sociology and seismology. Real-world datasets can be high-dimensional, contain structured missingness patterns, and measurement time points can be governed by an unknown process. We propose a flexible and efficient latent-variable model that is capable of addressing all these limitations.
  arXiv  Detail & Related papers  (2024-02-08T15:41:48Z)
• Equivariant Graph Neural Operator for Modeling 3D Dynamics [148.98826858078556]
  We propose Equivariant Graph Neural Operator (EGNO) to directly models dynamics as trajectories instead of just next-step prediction. EGNO explicitly learns the temporal evolution of 3D dynamics where we formulate the dynamics as a function over time and learn neural operators to approximate it. Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods.
  arXiv  Detail & Related papers  (2024-01-19T21:50:32Z)
• Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning [0.0]
  The deep latent force model (DLFM) is a deep Gaussian process with physics-informed kernels at each layer. We present empirical evidence of the capability of the DLFM to capture the dynamics present in highly nonlinear real-world time series data. We find that the DLFM is capable of achieving comparable performance to a range of non-physics-informed probabilistic models.
  arXiv  Detail & Related papers  (2023-11-24T19:55:57Z)
• Gait Recognition in the Wild with Multi-hop Temporal Switch [81.35245014397759]
  gait recognition in the wild is a more practical problem that has attracted the attention of the community of multimedia and computer vision. This paper presents a novel multi-hop temporal switch method to achieve effective temporal modeling of gait patterns in real-world scenes.
  arXiv  Detail & Related papers  (2022-09-01T10:46:09Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features [0.0]
  We present a novel, domain-agnostic approach to tackling this problem. We use compositions of physics-informed random features, derived from ordinary differential equations. We find that our approach achieves comparable performance to a number of other probabilistic models on benchmark regression tasks.
  arXiv  Detail & Related papers  (2021-06-10T17:55:13Z)
• Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernels [67.81799703916563]
  We introduce new techniques to formulate the problem as solving Fokker-Planck equation in a lower-dimensional latent space. Our proposed model consists of latent-distribution morphing, a generator and a parameterized Fokker-Planck kernel function.
  arXiv  Detail & Related papers  (2021-05-10T17:42:01Z)
• Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
  We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence. This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time. Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
  arXiv  Detail & Related papers  (2020-02-21T05:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.