Machine Learning Techniques for Data Reduction of Climate Applications

• URL: http://arxiv.org/abs/2405.00879v1
• Date: Wed, 1 May 2024 21:44:47 GMT
• Title: Machine Learning Techniques for Data Reduction of Climate Applications
• Authors: Xiao Li, Qian Gong, Jaemoon Lee, Scott Klasky, Anand Rangarajan, Sanjay Ranka,
• Abstract summary: We present a pipelined compression approach that first uses neural-network-based techniques to derive regions where QoI are highly likely to be present. We employ a Guaranteed Autoencoder (GAE) to compress data with differential error bounds. Results are presented for climate data generated from the E3SM Simulation model for downstream quantities such as tropical cyclone and atmospheric river detection and tracking.
• Score: 11.55089543867768
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientists conduct large-scale simulations to compute derived quantities-of-interest (QoI) from primary data. Often, QoI are linked to specific features, regions, or time intervals, such that data can be adaptively reduced without compromising the integrity of QoI. For many spatiotemporal applications, these QoI are binary in nature and represent presence or absence of a physical phenomenon. We present a pipelined compression approach that first uses neural-network-based techniques to derive regions where QoI are highly likely to be present. Then, we employ a Guaranteed Autoencoder (GAE) to compress data with differential error bounds. GAE uses QoI information to apply low-error compression to only these regions. This results in overall high compression ratios while still achieving downstream goals of simulation or data collections. Experimental results are presented for climate data generated from the E3SM Simulation model for downstream quantities such as tropical cyclone and atmospheric river detection and tracking. These results show that our approach is superior to comparable methods in the literature.

Related papers

• Causal Representation Learning in Temporal Data via Single-Parent Decoding [66.34294989334728]
  Scientific research often seeks to understand the causal structure underlying high-level variables in a system. Scientists typically collect low-level measurements, such as geographically distributed temperature readings. We propose a differentiable method, Causal Discovery with Single-parent Decoding, that simultaneously learns the underlying latents and a causal graph over them.
  arXiv  Detail & Related papers  (2024-10-09T15:57:50Z)
• Neural Compression of Atmospheric States [19.938380199858454]
  We propose a method for compressing atmospheric states using methods from the neural network. We show that both families of models satisfy the desiderata of small average error. We demonstrate compression ratios in excess of 1000x, with compression and decompression at a rate of approximately one second per global atmospheric state.
  arXiv  Detail & Related papers  (2024-07-16T12:38:39Z)
• Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling [16.890326773246414]
  We propose a downscaling method based on the Fourier neural operator. We show that our method significantly outperforms state-of-the-art convolutional and generative adversarial downscaling models. Overall, our work bridges the gap between simulation of a physical process and low-resolution output.
  arXiv  Detail & Related papers  (2023-05-23T18:25:08Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• Spatiotemporal modeling of European paleoclimate using doubly sparse Gaussian processes [61.31361524229248]
  We build on recent scale sparsetemporal GPs to reduce the computational burden. We successfully employ such a doubly sparse GP to construct a probabilistic model of paleoclimate.
  arXiv  Detail & Related papers  (2022-11-15T14:15:04Z)
• Advancing Reacting Flow Simulations with Data-Driven Models [50.9598607067535]
  Key to effective use of machine learning tools in multi-physics problems is to couple them to physical and computer models. The present chapter reviews some of the open opportunities for the application of data-driven reduced-order modeling of combustion systems.
  arXiv  Detail & Related papers  (2022-09-05T16:48:34Z)
• Pre-training via Denoising for Molecular Property Prediction [53.409242538744444]
  We describe a pre-training technique that utilizes large datasets of 3D molecular structures at equilibrium. Inspired by recent advances in noise regularization, our pre-training objective is based on denoising.
  arXiv  Detail & Related papers  (2022-05-31T22:28:34Z)
• A Physics-Informed Vector Quantized Autoencoder for Data Compression of Turbulent Flow [28.992515947961593]
  We apply a physics-informed Deep Learning technique based on vector quantization to generate a low-dimensional representation of data from turbulent flows. The accuracy of the model is assessed using statistical, comparison-based similarity and physics-based metrics. Our model can offer CR $=85$ with a mean square error (MSE) of $O(10-3)$, and predictions that faithfully reproduce the statistics of the flow, except at the very smallest scales.
  arXiv  Detail & Related papers  (2022-01-10T19:55:50Z)
• Convolutional generative adversarial imputation networks for spatio-temporal missing data in storm surge simulations [86.5302150777089]
  Generative Adversarial Imputation Nets (GANs) and GAN-based techniques have attracted attention as unsupervised machine learning methods. We name our proposed method as Con Conval Generative Adversarial Imputation Nets (Conv-GAIN)
  arXiv  Detail & Related papers  (2021-11-03T03:50:48Z)
• Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods [0.6875312133832078]
  We show that when the kernel of these emulators is also learned from data, the resulting data-driven models are faster than equation-based models. We see significant improvements over climatology and persistence based forecast techniques.
  arXiv  Detail & Related papers  (2021-02-13T19:57:33Z)
• Using Deep Learning to Explore Local Physical Similarity for Global-scale Bridging in Thermal-hydraulic Simulation [4.350727579753697]
  Current system thermal-hydraulic codes have limited credibility in simulating real plant conditions. This paper proposes a data-driven approach, Feature Similarity Measurement FFSM, to overcome these difficulties. Deep learning is applied to construct and explore the relationship between the local physical features and simulation errors.
  arXiv  Detail & Related papers  (2020-01-06T20:14:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.