Verification and Validation for Trustworthy Scientific Machine Learning

• URL: http://arxiv.org/abs/2502.15496v2
• Date: Fri, 25 Apr 2025 19:02:50 GMT
• Title: Verification and Validation for Trustworthy Scientific Machine Learning
• Authors: John D. Jakeman, Lorena A. Barba, Joaquim R. R. A. Martins, Thomas O'Leary-Roseberry,
• Abstract summary: The goal of this paper is to start a discussion on establishing consensus-based good practices for predictive SciML.<n>We identify key challenges in applying existing computational science and engineering guidelines, such as verification and validation protocols.<n>While centered on predictive applications, our 16 recommendations aim to help researchers conduct and document their modeling processes rigorously across all SciML domains.
• Score: 0.8749675983608172
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific machine learning (SciML) models are transforming many scientific disciplines. However, the development of good modeling practices to increase the trustworthiness of SciML has lagged behind its application, limiting its potential impact. The goal of this paper is to start a discussion on establishing consensus-based good practices for predictive SciML. We identify key challenges in applying existing computational science and engineering guidelines, such as verification and validation protocols, and provide recommendations to address these challenges. Our discussion focuses on predictive SciML, which uses machine learning models to learn, improve, and accelerate numerical simulations of physical systems. While centered on predictive applications, our 16 recommendations aim to help researchers conduct and document their modeling processes rigorously across all SciML domains.

Related papers

• Physics-Informed Inference Time Scaling via Simulation-Calibrated Scientific Machine Learning [5.728698570173857]
  High-dimensional partial differential equations (PDEs) pose significant computational challenges across fields ranging from quantum chemistry to economics and finance. Although scientific machine learning (SciML) techniques offer approximate solutions, they often suffer from bias and neglect crucial physical insights. We propose Simulation-Calibrated Scientific Machine Learning (SCa), a framework that dynamically refines and debiases the SCiML predictions during inference by enforcing the physical laws.
  arXiv  Detail & Related papers  (2025-04-22T18:01:45Z)
• LLM Post-Training: A Deep Dive into Reasoning Large Language Models [131.10969986056]
  Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications. Post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations.
  arXiv  Detail & Related papers  (2025-02-28T18:59:54Z)
• A Survey on Memory-Efficient Large-Scale Model Training in AI for Science [20.31466892935848]
  This survey reviews applications across scientific fields such as biology, medicine, chemistry, and meteorology.<n>We review memory-efficient training techniques for large language models (LLMs) based on the transformer architecture.<n>We demonstrate how tailored memory optimization methods can reduce storage needs while preserving prediction accuracy.
  arXiv  Detail & Related papers  (2025-01-21T03:06:30Z)
• MAPS: Advancing Multi-Modal Reasoning in Expert-Level Physical Science [62.96434290874878]
  Current Multi-Modal Large Language Models (MLLM) have shown strong capabilities in general visual reasoning tasks. We develop a new framework, named Multi-Modal Scientific Reasoning with Physics Perception and Simulation (MAPS) based on an MLLM. MAPS decomposes expert-level multi-modal reasoning task into physical diagram understanding via a Physical Perception Model (PPM) and reasoning with physical knowledge via a simulator.
  arXiv  Detail & Related papers  (2025-01-18T13:54:00Z)
• Geometry Matters: Benchmarking Scientific ML Approaches for Flow Prediction around Complex Geometries [23.111935712144277]
  Rapid and accurate simulations of fluid dynamics around complicated geometric bodies are critical in a variety of engineering and scientific applications. While scientific machine learning (SciML) has shown considerable promise, most studies in this field are limited to simple geometries. This paper addresses this gap by benchmarking diverse SciML models for fluid flow prediction over intricate geometries.
  arXiv  Detail & Related papers  (2024-12-31T00:23:15Z)
• Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey [51.87875066383221]
  This paper introduces fundamental concepts, traditional methods, and benchmark datasets, then examine the various roles Machine Learning plays in improving CFD. We highlight real-world applications of ML for CFD in critical scientific and engineering disciplines, including aerodynamics, combustion, atmosphere & ocean science, biology fluid, plasma, symbolic regression, and reduced order modeling. We draw the conclusion that ML is poised to significantly transform CFD research by enhancing simulation accuracy, reducing computational time, and enabling more complex analyses of fluid dynamics.
  arXiv  Detail & Related papers  (2024-08-22T07:33:11Z)
• Improving Molecular Modeling with Geometric GNNs: an Empirical Study [56.52346265722167]
  This paper focuses on the impact of different canonicalization methods, (2) graph creation strategies, and (3) auxiliary tasks, on performance, scalability and symmetry enforcement. Our findings and insights aim to guide researchers in selecting optimal modeling components for molecular modeling tasks.
  arXiv  Detail & Related papers  (2024-07-11T09:04:12Z)
• Reliable edge machine learning hardware for scientific applications [34.87898436984149]
  Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing. We discuss approaches to developing and validating reliable algorithms at the scientific edge under such strict latency, resource, power, and area requirements.
  arXiv  Detail & Related papers  (2024-06-27T20:45:08Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models [57.96527452844273]
  We introduce SciInstruct, a suite of scientific instructions for training scientific language models capable of college-level scientific reasoning. We curated a diverse and high-quality dataset encompassing physics, chemistry, math, and formal proofs. To verify the effectiveness of SciInstruct, we fine-tuned different language models with SciInstruct, i.e., ChatGLM3 (6B and 32B), Llama3-8B-Instruct, and Mistral-7B: MetaMath.
  arXiv  Detail & Related papers  (2024-01-15T20:22:21Z)
• Constructing Impactful Machine Learning Research for Astronomy: Best Practices for Researchers and Reviewers [0.0]
  Machine learning has rapidly become a tool of choice for the astronomical community. This paper provides a primer to the astronomical community on how to implement machine learning models and report their results.
  arXiv  Detail & Related papers  (2023-10-19T07:04:36Z)
• Differentiable modeling to unify machine learning and physical models and advance Geosciences [38.92849886903847]
  We outline the concepts, applicability, and significance of differentiable geoscientific modeling (DG) "Differentiable" refers to accurately and efficiently calculating gradients with respect to model variables. Preliminary evidence suggests DG offers better interpretability and causality than Machine Learning.
  arXiv  Detail & Related papers  (2023-01-10T15:24:14Z)
• Learning continuous models for continuous physics [94.42705784823997]
  We develop a test based on numerical analysis theory to validate machine learning models for science and engineering applications. Our results illustrate how principled numerical analysis methods can be coupled with existing ML training/testing methodologies to validate models for science and engineering applications.
  arXiv  Detail & Related papers  (2022-02-17T07:56:46Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)
• Towards the Development of Entropy-Based Anomaly Detection in an Astrophysics Simulation [0.2867517731896504]
  We present an anomaly problem which arises from a core-collapse supernovae simulation. We discuss strategies and early successes in applying anomaly detection techniques to this scientific simulation.
  arXiv  Detail & Related papers  (2020-09-05T01:43:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.