Uncovering Closed-form Governing Equations of Nonlinear Dynamics from
Videos
- URL: http://arxiv.org/abs/2106.04776v1
- Date: Wed, 9 Jun 2021 02:50:11 GMT
- Title: Uncovering Closed-form Governing Equations of Nonlinear Dynamics from
Videos
- Authors: Lele Luan, Yang Liu, Hao Sun
- Abstract summary: We introduce a novel end-to-end unsupervised deep learning framework to uncover the mathematical structure of equations that governs the dynamics of moving objects in videos.
Such an architecture consists of (1) an encoder-decoder network that learns low-dimensional spatial/pixel coordinates of the moving object, (2) a learnable Spatial-Physical Transformation component that creates mapping between the extracted spatial/pixel coordinates and the latent physical states of dynamics, and (3) a numerical integrator-based sparse regression module that uncovers the parsimonious closed-form governing equations of learned physical states.
- Score: 8.546520029145853
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Distilling analytical models from data has the potential to advance our
understanding and prediction of nonlinear dynamics. Although discovery of
governing equations based on observed system states (e.g., trajectory time
series) has revealed success in a wide range of nonlinear dynamics, uncovering
the closed-form equations directly from raw videos still remains an open
challenge. To this end, we introduce a novel end-to-end unsupervised deep
learning framework to uncover the mathematical structure of equations that
governs the dynamics of moving objects in videos. Such an architecture consists
of (1) an encoder-decoder network that learns low-dimensional spatial/pixel
coordinates of the moving object, (2) a learnable Spatial-Physical
Transformation component that creates mapping between the extracted
spatial/pixel coordinates and the latent physical states of dynamics, and (3) a
numerical integrator-based sparse regression module that uncovers the
parsimonious closed-form governing equations of learned physical states and,
meanwhile, serves as a constraint to the autoencoder. The efficacy of the
proposed method is demonstrated by uncovering the governing equations of a
variety of nonlinear dynamical systems depicted by moving objects in videos.
The resulting computational framework enables discovery of parsimonious
interpretable model in a flexible and accessible sensing environment where only
videos are available.
Related papers
- Vision-based Discovery of Nonlinear Dynamics for 3D Moving Target [11.102585080028945]
We propose a vision-based approach to automatically uncover governing equations of nonlinear dynamics for 3D moving targets via raw videos recorded by a set of cameras.
This framework is capable of effectively handling the challenges associated with measurement data, e.g., noise in the video, imprecise tracking of the target that causes data missing, etc.
arXiv Detail & Related papers (2024-04-27T11:13:55Z) - DynaVol: Unsupervised Learning for Dynamic Scenes through Object-Centric
Voxelization [67.85434518679382]
We present DynaVol, a 3D scene generative model that unifies geometric structures and object-centric learning.
The key idea is to perform object-centric voxelization to capture the 3D nature of the scene.
voxel features evolve over time through a canonical-space deformation function, forming the basis for global representation learning.
arXiv Detail & Related papers (2023-04-30T05:29:28Z) - NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos [82.74918564737591]
We present a method for learning 3D geometry and physics parameters of a dynamic scene from only a monocular RGB video input.
Experiments show that our method achieves superior mesh and video reconstruction of dynamic scenes compared to competing Neural Field approaches.
arXiv Detail & Related papers (2022-10-22T04:57:55Z) - Distilling Governing Laws and Source Input for Dynamical Systems from
Videos [13.084113582897965]
Distilling interpretable physical laws from videos has led to expanded interest in the computer vision community.
This paper introduces an end-to-end unsupervised deep learning framework to uncover the explicit governing equations of dynamics presented by moving object(s) based on recorded videos.
arXiv Detail & Related papers (2022-05-03T05:40:01Z) - Neural Implicit Representations for Physical Parameter Inference from a Single Video [49.766574469284485]
We propose to combine neural implicit representations for appearance modeling with neural ordinary differential equations (ODEs) for modelling physical phenomena.
Our proposed model combines several unique advantages: (i) Contrary to existing approaches that require large training datasets, we are able to identify physical parameters from only a single video.
The use of neural implicit representations enables the processing of high-resolution videos and the synthesis of photo-realistic images.
arXiv Detail & Related papers (2022-04-29T11:55:35Z) - Capturing Actionable Dynamics with Structured Latent Ordinary
Differential Equations [68.62843292346813]
We propose a structured latent ODE model that captures system input variations within its latent representation.
Building on a static variable specification, our model learns factors of variation for each input to the system, thus separating the effects of the system inputs in the latent space.
arXiv Detail & Related papers (2022-02-25T20:00:56Z) - Discovering Governing Equations from Partial Measurements with Deep
Delay Autoencoders [4.446017969073817]
A central challenge in data-driven model discovery is the presence of hidden, or latent, variables that are not directly measured but are dynamically important.
Here, we design a custom deep autoencoder network to learn a coordinate transformation from the delay embedded space into a new space.
We demonstrate this approach on the Lorenz, R"ossler, and Lotka-Volterra systems, learning dynamics from a single measurement variable.
arXiv Detail & Related papers (2022-01-13T18:48:16Z) - Constructing Neural Network-Based Models for Simulating Dynamical
Systems [59.0861954179401]
Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system.
This paper provides a survey of the different ways to construct models of dynamical systems using neural networks.
In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome.
arXiv Detail & Related papers (2021-11-02T10:51:42Z) - Physics-informed Spline Learning for Nonlinear Dynamics Discovery [8.546520029145853]
We propose a Physics-informed Spline Learning framework to discover parsimonious governing equations for nonlinear dynamics.
The framework is based on sparsely sampled noisy data.
The efficacy and superiority of the proposed method has been demonstrated by multiple well-known nonlinear dynamical systems.
arXiv Detail & Related papers (2021-05-05T23:32:43Z) - gradSim: Differentiable simulation for system identification and
visuomotor control [66.37288629125996]
We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering.
Our unified graph enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision.
arXiv Detail & Related papers (2021-04-06T16:32:01Z) - LagNetViP: A Lagrangian Neural Network for Video Prediction [12.645753197663584]
We introduce a video prediction model where the equations of motion are explicitly constructed from learned representations of the underlying physical quantities.
We demonstrate the efficacy of this approach for video prediction on image sequences rendered in modified OpenAI gym Pendulum-v0 and Acrobot environments.
arXiv Detail & Related papers (2020-10-24T16:50:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.