$\beta$-Variational autoencoders and transformers for reduced-order modelling of fluid flows

• URL: http://arxiv.org/abs/2304.03571v2
• Date: Wed, 15 Nov 2023 10:47:17 GMT
• Title: $\beta$-Variational autoencoders and transformers for reduced-order modelling of fluid flows
• Authors: Alberto Solera-Rico (1 and 2), Carlos Sanmiguel Vila (1 and 2), M. A. G\'omez (2), Yuning Wang (4), Abdulrahman Almashjary (3), Scott T. M. Dawson (3), Ricardo Vinuesa (4) (1: Aerospace Engineering Research Group, Universidad Carlos III de Madrid, Legan\'es, Spain 2: Subdirectorate General of Terrestrial Systems, Spanish National Institute for Aerospace Technology (INTA), San Mart\'in de la Vega, Spain 3: Mechanical, Materials, and Aerospace Engineering Department, Illinois Institute of Technology, Chicago, USA 4: FLOW, Engineering Mechanics, KTH Royal Institute of Technology, Stockholm, Sweden)
• Abstract summary: Variational autoencoder (VAE) architectures have the potential to develop reduced-order models (ROMs) for chaotic fluid flows. We propose a method for learning compact and near-orthogonal ROMs using a combination of a $beta$-VAE and a transformer.
• Score: 0.3644907558168858
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Variational autoencoder (VAE) architectures have the potential to develop reduced-order models (ROMs) for chaotic fluid flows. We propose a method for learning compact and near-orthogonal ROMs using a combination of a $\beta$-VAE and a transformer, tested on numerical data from a two-dimensional viscous flow in both periodic and chaotic regimes. The $\beta$-VAE is trained to learn a compact latent representation of the flow velocity, and the transformer is trained to predict the temporal dynamics in latent space. Using the $\beta$-VAE to learn disentangled representations in latent-space, we obtain a more interpretable flow model with features that resemble those observed in the proper orthogonal decomposition, but with a more efficient representation. Using Poincar\'e maps, the results show that our method can capture the underlying dynamics of the flow outperforming other prediction models. The proposed method has potential applications in other fields such as weather forecasting, structural dynamics or biomedical engineering.

Related papers

• Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis [97.54180451650122]
  We study the dynamics of training a shallow transformer on a task of recognizing co-occurrence of two designated words. We analyze the gradient flow dynamics of simultaneously training three attention matrices and a linear layer. We prove a novel property of the gradient flow, termed textitautomatic balancing of gradients, which enables the loss values of different samples to decrease almost at the same rate and further facilitates the proof of near minimum training loss.
  arXiv  Detail & Related papers  (2024-10-12T17:50:58Z)
• Towards aerodynamic surrogate modeling based on $β$-variational autoencoders [0.0]
  Surrogate models that combine dimensionality reduction and regression techniques are essential to reduce the need for costly high-fidelity computational fluid dynamics data. We propose a surrogate model based on latent space regression to predict pressure distributions on a transonic wing given the flight conditions: Mach number and angle of attack.
  arXiv  Detail & Related papers  (2024-08-09T09:43:10Z)
• KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter [49.85369344101118]
  We introduce KFD-NeRF, a novel dynamic neural radiance field integrated with an efficient and high-quality motion reconstruction framework based on Kalman filtering. Our key idea is to model the dynamic radiance field as a dynamic system whose temporally varying states are estimated based on two sources of knowledge: observations and predictions. Our KFD-NeRF demonstrates similar or even superior performance within comparable computational time and state-of-the-art view synthesis performance with thorough training.
  arXiv  Detail & Related papers  (2024-07-18T05:48:24Z)
• Koopman-Based Surrogate Modelling of Turbulent Rayleigh-Bénard Convection [4.248022697109535]
  We use a Koopman-inspired architecture called the Linear Recurrent Autoencoder Network (LRAN) for learning reduced-order dynamics in convection flows. A traditional fluid dynamics method, the Kernel Dynamic Mode Decomposition (KDMD) is used to compare the LRAN. We obtained more accurate predictions with the LRAN than with KDMD in the most turbulent setting.
  arXiv  Detail & Related papers  (2024-05-10T12:15:02Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• Differentiable Turbulence II [0.0]
  We develop a framework for integrating deep learning models into a generic finite element numerical scheme for solving the Navier-Stokes equations. We show that the learned closure can achieve accuracy comparable to traditional large eddy simulation on a finer grid that amounts to an equivalent speedup of 10x.
  arXiv  Detail & Related papers  (2023-07-25T14:27:49Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Forecasting subcritical cylinder wakes with Fourier Neural Operators [58.68996255635669]
  We apply a state-of-the-art operator learning technique to forecast the temporal evolution of experimentally measured velocity fields. We find that FNOs are capable of accurately predicting the evolution of experimental velocity fields throughout the range of Reynolds numbers tested.
  arXiv  Detail & Related papers  (2023-01-19T20:04:36Z)
• Emulating Spatio-Temporal Realizations of Three-Dimensional Isotropic Turbulence via Deep Sequence Learning Models [24.025975236316842]
  We use a data-driven approach to model a three-dimensional turbulent flow using cutting-edge Deep Learning techniques. The accuracy of the model is assessed using statistical and physics-based metrics.
  arXiv  Detail & Related papers  (2021-12-07T03:33:39Z)
• Echo State Network for two-dimensional turbulent moist Rayleigh-B\'enard convection [0.0]
  We apply an echo state network to approximate the evolution of moist Rayleigh-B'enard convection. We conclude that our model is capable of learning complex dynamics.
  arXiv  Detail & Related papers  (2021-01-27T11:27:16Z)
• Machine learning for rapid discovery of laminar flow channel wall modifications that enhance heat transfer [56.34005280792013]
  We present a combination of accurate numerical simulations of arbitrary, flat, and non-flat channels and machine learning models predicting drag coefficient and Stanton number. We show that convolutional neural networks (CNN) can accurately predict the target properties at a fraction of the time of numerical simulations.
  arXiv  Detail & Related papers  (2021-01-19T16:14:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.