Predicting Physics in Mesh-reduced Space with Temporal Attention

• URL: http://arxiv.org/abs/2201.09113v1
• Date: Sat, 22 Jan 2022 18:32:54 GMT
• Title: Predicting Physics in Mesh-reduced Space with Temporal Attention
• Authors: Xu Han and Han Gao and Tobias Pffaf and Jian-Xun Wang and Li-Ping Liu
• Abstract summary: We propose a new method that captures long-term dependencies through a transformer-style temporal attention model. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks. We believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.
• Score: 15.054026802351146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes. However, due to their short temporal attention span, these models suffer from error accumulation and drift. In this paper, we propose a new method that captures long-term dependencies through a transformer-style temporal attention model. We introduce an encoder-decoder structure to summarize features and create a compact mesh representation of the system state, to allow the temporal model to operate on a low-dimensional mesh representations in a memory efficient manner. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We demonstrate stable rollouts without the need for training noise and show perfectly phase-stable predictions even for very long sequences. More broadly, we believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.

Related papers

• Temporally-Consistent Koopman Autoencoders for Forecasting Dynamical Systems [42.6886113798806]
  We introduce the Temporally-Consistent Koopman Autoencoder (tcKAE) tcKAE generates accurate long-term predictions even with constrained and noisy training data. We demonstrate tcKAE's superior performance over state-of-the-art KAE models across a variety of test cases.
  arXiv  Detail & Related papers  (2024-03-19T00:48:25Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• Koopman Invertible Autoencoder: Leveraging Forward and Backward Dynamics for Temporal Modeling [13.38194491846739]
  We propose a novel machine learning model based on Koopman operator theory, which we call Koopman Invertible Autoencoders (KIA) KIA captures the inherent characteristic of the system by modeling both forward and backward dynamics in the infinite-dimensional Hilbert space. This enables us to efficiently learn low-dimensional representations, resulting in more accurate predictions of long-term system behavior.
  arXiv  Detail & Related papers  (2023-09-19T03:42:55Z)
• Learning Nonlinear Projections for Reduced-Order Modeling of Dynamical Systems using Constrained Autoencoders [0.0]
  We introduce a class of nonlinear projections described by constrained autoencoder neural networks in which both the manifold and the projection fibers are learned from data. Our architecture uses invertible activation functions and biorthogonal weight matrices to ensure that the encoder is a left inverse of the decoder. We also introduce new dynamics-aware cost functions that promote learning of oblique projection fibers that account for fast dynamics and nonnormality.
  arXiv  Detail & Related papers  (2023-07-28T04:01:48Z)
• Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systems [14.553972457854517]
  We present a data-driven, space-time continuous framework to learn surrogatemodels for complex physical systems. We leverage the expressive power of the network and aspecially designed consistency-inducing regularization to obtain latent trajectories that are both low-dimensional and smooth.
  arXiv  Detail & Related papers  (2023-01-25T03:06:03Z)
• Data-driven low-dimensional dynamic model of Kolmogorov flow [0.0]
  Reduced order models (ROMs) that capture flow dynamics are of interest for decreasing computational costs for simulation. This work presents a data-driven framework for minimal-dimensional models that effectively capture the dynamics and properties of the flow. We apply this to Kolmogorov flow in a regime consisting of chaotic and intermittent behavior.
  arXiv  Detail & Related papers  (2022-10-29T23:05:39Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.