FlowBERT: Prompt-tuned BERT for variable flow field prediction

• URL: http://arxiv.org/abs/2506.08021v1
• Date: Tue, 20 May 2025 02:25:38 GMT
• Title: FlowBERT: Prompt-tuned BERT for variable flow field prediction
• Authors: Weihao Zou, Weibing Feng, Pin Wu,
• Abstract summary: This study proposes a universal flow field prediction framework based on knowledge transfer from large language model (LLM)<n>Our approach reduces prediction time to seconds while maintaining over 90% accuracy.<n>The developed knowledge transfer paradigm establishes a new direction for rapid fluid dynamics prediction.
• Score: 0.5222978725954347
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study proposes a universal flow field prediction framework based on knowledge transfer from large language model (LLM), addressing the high computational costs of traditional computational fluid dynamics (CFD) methods and the limited cross-condition transfer capability of existing deep learning models. The framework innovatively integrates Proper Orthogonal Decomposition (POD) dimensionality reduction with fine-tuning strategies for pretrained LLM, where POD facilitates compressed representation of flow field features while the fine-tuned model learns to encode system dynamics in state space. To enhance the model's adaptability to flow field data, we specifically designed fluid dynamics-oriented text templates that improve predictive performance through enriched contextual semantic information. Experimental results demonstrate that our framework outperforms conventional Transformer models in few-shot learning scenarios while exhibiting exceptional generalization across various inflow conditions and airfoil geometries. Ablation studies reveal the contributions of key components in the FlowBERT architecture. Compared to traditional Navier-Stokes equation solvers requiring hours of computation, our approach reduces prediction time to seconds while maintaining over 90% accuracy. The developed knowledge transfer paradigm establishes a new direction for rapid fluid dynamics prediction, with potential applications extending to aerodynamic optimization, flow control, and other engineering domains.

Related papers

• FlowState: Sampling Rate Invariant Time Series Forecasting [0.7999703756441756]
  FlowState is a novel time series foundation model (TSFM) architecture.<n>It inherently generalizes across all possible temporal resolutions and dynamically adjusts the forecasting horizons.<n>It is state-of-the-art for the GIFT-ZS and the Chronos-ZS benchmarks.
  arXiv  Detail & Related papers  (2025-08-07T11:30:26Z)
• Autoregressive regularized score-based diffusion models for multi-scenarios fluid flow prediction [0.0]
  We propose a conditional score-based diffusion model for multi-scenarios fluid flow prediction.<n>Our model integrates an energy constraint rooted in the statistical properties of turbulent flows.<n>It achieves accurate results across multiple scenarios while preserving key physical and statistical properties.
  arXiv  Detail & Related papers  (2025-05-30T02:37:05Z)
• An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training [50.71892161377806]
  DFIT-OccWorld is an efficient 3D occupancy world model that leverages decoupled dynamic flow and image-assisted training strategy.<n>Our model forecasts future dynamic voxels by warping existing observations using voxel flow, whereas static voxels are easily obtained through pose transformation.
  arXiv  Detail & Related papers  (2024-12-18T12:10:33Z)
• Hierarchically Disentangled Recurrent Network for Factorizing System Dynamics of Multi-scale Systems: An application on Hydrological Systems [4.634606500665259]
  We propose a novel hierarchical recurrent neural architecture that factorizes the system dynamics at multiple temporal scales.<n> Experiments on several catchments from the National Weather Service North Central River Forecast Center show that FHNN outperforms standard baselines.<n>We show that FHNN can maintain accuracy even with limited training data through effective pre-training strategies and training global models.
  arXiv  Detail & Related papers  (2024-07-29T16:25:43Z)
• AdaOcc: Adaptive Forward View Transformation and Flow Modeling for 3D Occupancy and Flow Prediction [56.72301849123049]
  We present our solution for the Vision-Centric 3D Occupancy and Flow Prediction track in the nuScenes Open-Occ dataset challenge at CVPR 2024. Our innovative approach involves a dual-stage framework that enhances 3D occupancy and flow predictions by incorporating adaptive forward view transformation and flow modeling. Our method combines regression with classification to address scale variations in different scenes, and leverages predicted flow to warp current voxel features to future frames, guided by future frame ground truth.
  arXiv  Detail & Related papers  (2024-07-01T16:32:15Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Emulating Spatio-Temporal Realizations of Three-Dimensional Isotropic Turbulence via Deep Sequence Learning Models [24.025975236316842]
  We use a data-driven approach to model a three-dimensional turbulent flow using cutting-edge Deep Learning techniques. The accuracy of the model is assessed using statistical and physics-based metrics.
  arXiv  Detail & Related papers  (2021-12-07T03:33:39Z)
• Transfer learning to improve streamflow forecasts in data sparse regions [0.0]
  We study the methodology behind Transfer Learning (TL) through fine-tuning and parameter transferring for better generalization performance of streamflow prediction in data-sparse regions. We propose a standard recurrent neural network in the form of Long Short-Term Memory (LSTM) to fit on a sufficiently large source domain dataset. We present a methodology to implement transfer learning approaches for hydrologic applications by separating the spatial and temporal components of the model and training the model to generalize.
  arXiv  Detail & Related papers  (2021-12-06T14:52:53Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.