An Extensible Benchmark Suite for Learning to Simulate Physical Systems
- URL: http://arxiv.org/abs/2108.07799v1
- Date: Mon, 9 Aug 2021 17:39:09 GMT
- Title: An Extensible Benchmark Suite for Learning to Simulate Physical Systems
- Authors: Karl Otness, Arvi Gjoka, Joan Bruna, Daniele Panozzo, Benjamin
Peherstorfer, Teseo Schneider, Denis Zorin
- Abstract summary: We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols.
We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
- Score: 60.249111272844374
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Simulating physical systems is a core component of scientific computing,
encompassing a wide range of physical domains and applications. Recently, there
has been a surge in data-driven methods to complement traditional numerical
simulations methods, motivated by the opportunity to reduce computational costs
and/or learn new physical models leveraging access to large collections of
data. However, the diversity of problem settings and applications has led to a
plethora of approaches, each one evaluated on a different setup and with
different evaluation metrics. We introduce a set of benchmark problems to take
a step towards unified benchmarks and evaluation protocols. We propose four
representative physical systems, as well as a collection of both widely used
classical time integrators and representative data-driven methods
(kernel-based, MLP, CNN, nearest neighbors). Our framework allows evaluating
objectively and systematically the stability, accuracy, and computational
efficiency of data-driven methods. Additionally, it is configurable to permit
adjustments for accommodating other learning tasks and for establishing a
foundation for future developments in machine learning for scientific
computing.
Related papers
- Machine Learning for predicting chaotic systems [0.0]
We show that well-tuned simple methods, as well as untuned baseline methods, often outperform state-of-the-art deep learning models.
These findings underscore the importance of matching prediction methods to data characteristics and available computational resources.
arXiv Detail & Related papers (2024-07-29T16:34:47Z) - Interpretable Meta-Learning of Physical Systems [4.343110120255532]
Recent meta-learning methods rely on black-box neural networks, resulting in high computational costs and limited interpretability.
We argue that multi-environment generalization can be achieved using a simpler learning model, with an affine structure with respect to the learning task.
We demonstrate the competitive generalization performance and the low computational cost of our method by comparing it to state-of-the-art algorithms on physical systems.
arXiv Detail & Related papers (2023-12-01T10:18:50Z) - DynaBench: A benchmark dataset for learning dynamical systems from
low-resolution data [3.8695554579762814]
We introduce a novel simulated benchmark dataset, DynaBench, for learning dynamical systems directly from sparse data.
The dataset focuses on predicting the evolution of a dynamical system from low-resolution, unstructured measurements.
arXiv Detail & Related papers (2023-06-09T10:42:32Z) - PDEBENCH: An Extensive Benchmark for Scientific Machine Learning [20.036987098901644]
We introduce PDEBench, a benchmark suite of time-dependent simulation tasks based on Partial Differential Equations (PDEs)
PDEBench comprises both code and data to benchmark the performance of novel machine learning models against both classical numerical simulations and machine learning baselines.
arXiv Detail & Related papers (2022-10-13T17:03:36Z) - Learning Physical Concepts in Cyber-Physical Systems: A Case Study [72.74318982275052]
We provide an overview of the current state of research regarding methods for learning physical concepts in time series data.
We also analyze the most important methods from the current state of the art using the example of a three-tank system.
arXiv Detail & Related papers (2021-11-28T14:24:52Z) - Constructing Neural Network-Based Models for Simulating Dynamical
Systems [59.0861954179401]
Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system.
This paper provides a survey of the different ways to construct models of dynamical systems using neural networks.
In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome.
arXiv Detail & Related papers (2021-11-02T10:51:42Z) - Using Data Assimilation to Train a Hybrid Forecast System that Combines
Machine-Learning and Knowledge-Based Components [52.77024349608834]
We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is noisy partial measurements.
We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.
arXiv Detail & Related papers (2021-02-15T19:56:48Z) - Model-Based Deep Learning [155.063817656602]
Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques.
Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance.
We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
arXiv Detail & Related papers (2020-12-15T16:29:49Z) - A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world.
We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms.
Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
arXiv Detail & Related papers (2020-11-17T22:24:26Z) - Learning Similarity Metrics for Numerical Simulations [29.39625644221578]
We propose a neural network-based approach that computes a stable and generalizing metric (LSiM) to compare data from a variety of numerical simulation sources.
Our method employs a Siamese network architecture that is motivated by the mathematical properties of a metric.
arXiv Detail & Related papers (2020-02-18T20:11:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.