BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics

• URL: http://arxiv.org/abs/2501.18972v2
• Date: Wed, 30 Apr 2025 03:02:41 GMT
• Title: BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics
• Abstract summary: We introduce BCAT, a PDE foundation model designed for autoregressive prediction of solutions to two dimensional fluid dynamics problems.<n>Our approach uses a block causal architecture to model next frame predictions, leveraging previous frames as contextual priors rather than relying solely on sub-frames or pixel-based inputs commonly used in image generation methods.
• Score: 4.027470328330335
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce BCAT, a PDE foundation model designed for autoregressive prediction of solutions to two dimensional fluid dynamics problems. Our approach uses a block causal transformer architecture to model next frame predictions, leveraging previous frames as contextual priors rather than relying solely on sub-frames or pixel-based inputs commonly used in image generation methods. This block causal framework more effectively captures the spatial dependencies inherent in nonlinear spatiotemporal dynamics and physical phenomena. In an ablation study, next frame prediction demonstrated a 3.5x accuracy improvement over next token prediction. BCAT is trained on a diverse range of fluid dynamics datasets, including incompressible and compressible Navier-Stokes equations across various geometries and parameter regimes, as well as the shallow-water equations. The model's performance was evaluated on 6 distinct downstream prediction tasks and tested on about 8K trajectories to measure robustness on a variety of fluid dynamics simulations. BCAT achieved an average relative error of 1.18% across all evaluation tasks, outperforming prior approaches on standard benchmarks. With fine-tuning on a turbulence dataset, we show that the method adapts to new settings with more than 40% better accuracy over prior methods.

Related papers

• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• Prediction of Steady-State Flow through Porous Media Using Machine Learning Models [3.2250618460602705]
  Flow through porous media is a crucial step in the topology optimisation of cold plates.<n>Traditional computational fluid dynamics methods are often prohibitively expensive for large and complex geometries.<n>We develop a machine-learning framework for predicting steady-state flow through porous media governed by the Navier-Stokes-Brinkman equations.
  arXiv  Detail & Related papers  (2026-03-06T14:28:46Z)
• PDE foundation model-accelerated inverse estimation of system parameters in inertial confinement fusion [36.885866084809926]
  We study an inverse problem in inertial confinement fusion (ICF): estimating system parameters (inputs) from multi-modal, snapshot-style observations (outputs)<n>We finetune the PDE foundation model and train a lightweight task-specific head to jointly reconstruct hyperspectral images and regress system parameters.<n>Experiments show consistent improvements in both reconstruction and regression losses as the amount of training data increases.
  arXiv  Detail & Related papers  (2026-03-04T21:07:43Z)
• Out-of-distribution transfer of PDE foundation models to material dynamics under extreme loading [86.6550968435969]
  Most PDE foundation models are pretrained and fine-tuned on fluid-centric benchmarks.<n>We benchmark out-of-distribution transfer on two discontinuity-dominated regimes in which shocks, evolving interfaces, and fracture produce highly non-smooth fields.<n>We evaluate two open-source PDE foundation models, POSEIDON and MORPH, and compare fine-tuning from pretrained weights against training from scratch across training-set sizes to quantify sample efficiency under distribution shift.
  arXiv  Detail & Related papers  (2026-03-04T18:19:35Z)
• Surrogate models for Rock-Fluid Interaction: A Grid-Size-Invariant Approach [0.3518016233072556]
  We develop eight surrogate models for predicting the fluid flow in porous media.<n>Four of these are reduced-order models (ROM) based on one neural network for compression and another for prediction.<n>We show that the grid-size-invariant approach is a reliable way to reduce memory consumption during training.
  arXiv  Detail & Related papers  (2026-02-25T18:34:03Z)
• Visual Autoregressive Modelling for Monocular Depth Estimation [69.01449528371916]
  We propose a monocular depth estimation method based on visual autoregressive ( VAR) priors.<n>Our method adapts a large-scale text-to-image VAR model and introduces a scale-wise conditional upsampling mechanism.<n>We report state-of-the-art performance in indoor benchmarks under constrained training conditions, and strong performance when applied to outdoor datasets.
  arXiv  Detail & Related papers  (2025-12-27T17:08:03Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• PhysicsCorrect: A Training-Free Approach for Stable Neural PDE Simulations [4.7903561901859355]
  We present PhysicsCorrect, a training-free correction framework that enforces PDE consistency at each prediction step.<n>Our key innovation is an efficient caching strategy that precomputes the Jacobian and its pseudoinverse during an offline warm-up phase.<n>Across three representative PDE systems, PhysicsCorrect reduces prediction errors by up to 100x while adding negligible inference time.
  arXiv  Detail & Related papers  (2025-07-03T01:22:57Z)
• Solving Inverse Problems with FLAIR [59.02385492199431]
  Flow-based latent generative models are able to generate images with remarkable quality, even enabling text-to-image generation.<n>We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• A Comparison of Parametric Dynamic Mode Decomposition Algorithms for Thermal-Hydraulics Applications [0.5033155053523042]
  This work contributes to the research field of learning models from available data by comparing different algorithms deployed in the field. To this aim, three different thermal-hydraulics problems are considered, whose datasets are, respectively, obtained with the FEniCS finite element solver and retrieved from the CFDbench dataset.
  arXiv  Detail & Related papers  (2025-03-31T15:23:22Z)
• Physics-constrained coupled neural differential equations for one dimensional blood flow modeling [0.3749861135832073]
  Computational cardiovascular flow modeling plays a crucial role in understanding blood flow dynamics.<n>Traditional 1D models based on finite element methods (FEM) often lack accuracy compared to 3D averaged solutions.<n>This study introduces a novel physics-constrained machine learning technique that enhances the accuracy of 1D blood flow models.
  arXiv  Detail & Related papers  (2024-11-08T15:22:20Z)
• Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems [49.11170948406405]
  State-of-the-art in automatic parameter estimation from video is addressed by training supervised deep networks on large datasets. We propose a method to estimate the physical parameters of any known, continuous governing equation from single videos.
  arXiv  Detail & Related papers  (2024-10-02T09:44:54Z)
• PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics [3.770825791788951]
  We propose a zero-shot multimodal PDE foundational model for simultaneous prediction of heterogeneous two-dimensional physical systems. These systems include shallow water equations and the Navier-Stokes equations with incompressible and compressible flow.
  arXiv  Detail & Related papers  (2024-09-15T18:20:15Z)
• Hierarchically Disentangled Recurrent Network for Factorizing System Dynamics of Multi-scale Systems: An application on Hydrological Systems [4.634606500665259]
  We propose a novel hierarchical recurrent neural architecture that factorizes the system dynamics at multiple temporal scales. Experiments on several catchments from the National Weather Service North Central River Forecast Center show that FHNN outperforms standard baselines. We show that FHNN can maintain accuracy even with limited training data through effective pre-training strategies and training global models.
  arXiv  Detail & Related papers  (2024-07-29T16:25:43Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• AdaOcc: Adaptive Forward View Transformation and Flow Modeling for 3D Occupancy and Flow Prediction [56.72301849123049]
  We present our solution for the Vision-Centric 3D Occupancy and Flow Prediction track in the nuScenes Open-Occ dataset challenge at CVPR 2024. Our innovative approach involves a dual-stage framework that enhances 3D occupancy and flow predictions by incorporating adaptive forward view transformation and flow modeling. Our method combines regression with classification to address scale variations in different scenes, and leverages predicted flow to warp current voxel features to future frames, guided by future frame ground truth.
  arXiv  Detail & Related papers  (2024-07-01T16:32:15Z)
• Graph Neural Networks for Pressure Estimation in Water Distribution Systems [44.99833362998488]
  Pressure and flow estimation in Water Distribution Networks (WDN) allows water management companies to optimize their control operations. We combine physics-based modeling and Graph Neural Networks (GNN), a data-driven approach, to address the pressure estimation problem. Our GNN-based model estimates the pressure of a large-scale WDN in The Netherlands with a MAE of 1.94mH$$O and a MAPE of 7%.
  arXiv  Detail & Related papers  (2023-11-17T15:30:12Z)
• Forecasting through deep learning and modal decomposition in two-phase concentric jets [2.362412515574206]
  This work aims to improve fuel chamber injectors' performance in turbofan engines. It requires the development of models that allow real-time prediction and improvement of the fuel/air mixture.
  arXiv  Detail & Related papers  (2022-12-24T12:59:41Z)
• Human Trajectory Prediction via Neural Social Physics [63.62824628085961]
  Trajectory prediction has been widely pursued in many fields, and many model-based and model-free methods have been explored. We propose a new method combining both methodologies based on a new Neural Differential Equation model. Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters.
  arXiv  Detail & Related papers  (2022-07-21T12:11:18Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction [94.25865526414717]
  This paper considers a new problem of adapting a pre-trained model of human mesh reconstruction to out-of-domain streaming videos. We propose Bilevel Online Adaptation, which divides the optimization process of overall multi-objective into two steps of weight probe and weight update in a training. We demonstrate that BOA leads to state-of-the-art results on two human mesh reconstruction benchmarks.
  arXiv  Detail & Related papers  (2021-03-30T15:47:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.