Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD

• URL: http://arxiv.org/abs/2511.01830v1
• Date: Mon, 03 Nov 2025 18:37:38 GMT
• Title: Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD
• Authors: Paul Setinek, Gianluca Galletti, Johannes Brandstetter,
• Abstract summary: Scaling laws describe how model performance grows with data, parameters and compute.<n>We investigate this trade-off between data fidelity and cost in neural surrogates using low- and high-fidelity simulations.<n>Our experiments reveal compute-performance scaling behavior and exhibit budget-dependent optimal fidelity mixes for the given dataset configuration.
• Score: 21.38912245186567
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling laws describe how model performance grows with data, parameters and compute. While large datasets can usually be collected at relatively low cost in domains such as language or vision, scientific machine learning is often limited by the high expense of generating training data through numerical simulations. However, by adjusting modeling assumptions and approximations, simulation fidelity can be traded for computational cost, an aspect absent in other domains. We investigate this trade-off between data fidelity and cost in neural surrogates using low- and high-fidelity Reynolds-Averaged Navier-Stokes (RANS) simulations. Reformulating classical scaling laws, we decompose the dataset axis into compute budget and dataset composition. Our experiments reveal compute-performance scaling behavior and exhibit budget-dependent optimal fidelity mixes for the given dataset configuration. These findings provide the first study of empirical scaling laws for multi-fidelity neural surrogate datasets and offer practical considerations for compute-efficient dataset generation in scientific machine learning.

Related papers

• Neural Scaling Laws for Boosted Jet Tagging [0.22399170518036912]
  scaling compute, through joint increases in model capacity and dataset size, is the primary driver of performance in modern machine learning.<n>We derive compute optimal scaling laws and identify an effective performance limit that can be consistently approached through increased compute.<n>We then study how the scaling coefficients and performance limits vary with the choice of input features and particle multiplicity.
  arXiv  Detail & Related papers  (2026-02-17T18:13:01Z)
• Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixtures [32.89034139737846]
  Large language models (LLMs) are built on datasets that blend real and synthetic data.<n> synthetic data offers scalability and cost-efficiency, but it often introduces systematic distributional discrepancies.<n>We propose an effective yet efficient data valuation method that scales to large-scale datasets.
  arXiv  Detail & Related papers  (2025-11-17T17:53:12Z)
• Transfer learning for multifidelity simulation-based inference in cosmology [0.0]
  Pre-training on dark-matter-only $N$-body simulations reduces the required number of high-fidelity hydrodynamical simulations by a factor between $8$ and $15$.<n>By leveraging cheaper simulations, our approach enables performant and accurate inference on high-fidelity models while substantially reducing computational costs.
  arXiv  Detail & Related papers  (2025-05-27T14:04:30Z)
• Multifidelity Simulation-based Inference for Computationally Expensive Simulators [7.065679767112407]
  We introduce MF-(TS)NPE, a multifidelity approach to neural posterior estimation that uses transfer learning to leverage inexpensive low-fidelity simulations.<n>We further improve simulation efficiency by introducing A-MF-TSNPE, a sequential variant that uses an acquisition function targeting the predictive uncertainty of the density estimator.
  arXiv  Detail & Related papers  (2025-02-12T13:59:22Z)
• Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws [59.03420759554073]
  We introduce Adaptive Data Optimization (ADO), an algorithm that optimize data distributions in an online fashion, concurrent with model training. ADO does not require external knowledge, proxy models, or modifications to the model update. ADO uses per-domain scaling laws to estimate the learning potential of each domain during training and adjusts the data mixture accordingly.
  arXiv  Detail & Related papers  (2024-10-15T17:47:44Z)
• Scaling Law of Sim2Real Transfer Learning in Expanding Computational Materials Databases for Real-World Predictions [13.20562263181952]
  Fine-tuning a predictor pretrained on a computational database to a real system can result in models with outstanding generalization capabilities. This study demonstrates the scaling law of simulation-to-real (Sim2Real) transfer learning for several machine learning tasks in materials science.
  arXiv  Detail & Related papers  (2024-08-07T18:47:58Z)
• Generative Expansion of Small Datasets: An Expansive Graph Approach [13.053285552524052]
  We introduce an Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples. An autoencoder with self-attention layers and optimal transport refines distributional consistency. Results show comparable performance, demonstrating the model's potential to augment training data effectively.
  arXiv  Detail & Related papers  (2024-06-25T02:59:02Z)
• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Scaling Laws for Sparsely-Connected Foundation Models [70.41266138010657]
  We explore the impact of parameter sparsity on the scaling behavior of Transformers trained on massive datasets. We identify the first scaling law describing the relationship between weight sparsity, number of non-zero parameters, and amount of training data.
  arXiv  Detail & Related papers  (2023-09-15T16:29:27Z)
• A Solvable Model of Neural Scaling Laws [72.8349503901712]
  Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws. We propose a statistical model -- a joint generative data model and random feature model -- that captures this neural scaling phenomenology. Key findings are the manner in which the power laws that occur in the statistics of natural datasets are extended by nonlinear random feature maps.
  arXiv  Detail & Related papers  (2022-10-30T15:13:18Z)
• Multi-fidelity Hierarchical Neural Processes [79.0284780825048]
  Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling. We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2022-06-10T04:54:13Z)
• Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training [126.49572353148262]
  We study two factors in neural network training: data parallelism and sparsity. Despite their promising benefits, understanding of their effects on neural network training remains elusive.
  arXiv  Detail & Related papers  (2020-03-25T10:49:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.