Evaluating Loss Functions and Learning Data Pre-Processing for Climate Downscaling Deep Learning Models

• URL: http://arxiv.org/abs/2306.11144v1
• Date: Mon, 19 Jun 2023 19:58:42 GMT
• Title: Evaluating Loss Functions and Learning Data Pre-Processing for Climate Downscaling Deep Learning Models
• Authors: Xingying Huang
• Abstract summary: We study the effects of loss functions and non-linear data pre-processing methods for deep learning models in the context of climate downscaling. Our findings reveal that L1 loss and L2 loss perform similarly on some more balanced data like temperature data while for some imbalanced data like precipitation data, L2 loss performs significantly better than L1 loss.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep learning models have gained popularity in climate science, following their success in computer vision and other domains. For instance, researchers are increasingly employing deep learning techniques for downscaling climate data, drawing inspiration from image super-resolution models. However, there are notable differences between image data and climate data. While image data typically falls within a specific range (e.g., [0, 255]) and exhibits a relatively uniform or normal distribution, climate data can possess arbitrary value ranges and highly uneven distributions, such as precipitation data. This non-uniform distribution presents challenges when attempting to directly apply existing computer vision models to climate science tasks. Few studies have addressed this issue thus far. In this study, we explore the effects of loss functions and non-linear data pre-processing methods for deep learning models in the context of climate downscaling. We employ a climate downscaling experiment as an example to evaluate these factors. Our findings reveal that L1 loss and L2 loss perform similarly on some more balanced data like temperature data while for some imbalanced data like precipitation data, L2 loss performs significantly better than L1 loss. Additionally, we propose an approach to automatically learn the non-linear pre-processing function, which further enhances model accuracy and achieves the best results.

Related papers

• Causal Representation Learning in Temporal Data via Single-Parent Decoding [66.34294989334728]
  Scientific research often seeks to understand the causal structure underlying high-level variables in a system. Scientists typically collect low-level measurements, such as geographically distributed temperature readings. We propose a differentiable method, Causal Discovery with Single-parent Decoding, that simultaneously learns the underlying latents and a causal graph over them.
  arXiv  Detail & Related papers  (2024-10-09T15:57:50Z)
• Mind the Gap Between Synthetic and Real: Utilizing Transfer Learning to Probe the Boundaries of Stable Diffusion Generated Data [2.6016285265085526]
  Student models show a significant drop in accuracy compared to models trained on real data. By training these layers using either real or synthetic data, we reveal that the drop mainly stems from the model's final layers. Our results suggest an improved trade-off between the amount of real training data used and the model's accuracy.
  arXiv  Detail & Related papers  (2024-05-06T07:51:13Z)
• Generating High-Resolution Regional Precipitation Using Conditional Diffusion Model [7.784934642915291]
  This paper presents a deep generative model for downscaling climate data, specifically precipitation on a regional scale. We employ a denoising diffusion probabilistic model conditioned on multiple LR climate variables. Our results demonstrate significant improvements over existing baselines, underscoring the effectiveness of the conditional diffusion model in downscaling climate data.
  arXiv  Detail & Related papers  (2023-12-12T09:39:52Z)
• Exploring the Effectiveness of Dataset Synthesis: An application of Apple Detection in Orchards [68.95806641664713]
  We explore the usability of Stable Diffusion 2.1-base for generating synthetic datasets of apple trees for object detection. We train a YOLOv5m object detection model to predict apples in a real-world apple detection dataset. Results demonstrate that the model trained on generated data is slightly underperforming compared to a baseline model trained on real-world images.
  arXiv  Detail & Related papers  (2023-06-20T09:46:01Z)
• An evaluation of deep learning models for predicting water depth evolution in urban floods [59.31940764426359]
  We compare different deep learning models for prediction of water depth at high spatial resolution. Deep learning models are trained to reproduce the data simulated by the CADDIES cellular-automata flood model. Our results show that the deep learning models present in general lower errors compared to the other methods.
  arXiv  Detail & Related papers  (2023-02-20T16:08:54Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation [151.70234052015948]
  We propose a novel approach that encourages the optimization algorithm to seek a flat trajectory. We show that the weights trained on synthetic data are robust against the accumulated errors perturbations with the regularization towards the flat trajectory. Our method, called Flat Trajectory Distillation (FTD), is shown to boost the performance of gradient-matching methods by up to 4.7%.
  arXiv  Detail & Related papers  (2022-11-20T15:49:11Z)
• Hard-Constrained Deep Learning for Climate Downscaling [30.280862393706542]
  High-resolution climate and weather data is important to inform long-term decisions on climate adaptation and mitigation. Forecasting models are limited by computational costs and, therefore, often generate coarse-resolution predictions. Statistical downscaling, including super-resolution methods from deep learning, can provide an efficient method of upsampling low-resolution data.
  arXiv  Detail & Related papers  (2022-08-08T16:54:01Z)
• Improved Loss Function-Based Prediction Method of Extreme Temperatures in Greenhouses [3.4893854610476267]
  The prediction of extreme greenhouse temperatures to which crops are susceptible is essential in the field of greenhouse planting. Due to the lack of extreme temperature data in datasets, it is challenging for models to accurately predict it. We propose an improved loss function, which is suitable for a variety of machine learning models.
  arXiv  Detail & Related papers  (2021-11-02T04:33:15Z)
• Taxonomizing local versus global structure in neural network loss landscapes [60.206524503782006]
  We show that the best test accuracy is obtained when the loss landscape is globally well-connected. We also show that globally poorly-connected landscapes can arise when models are small or when they are trained to lower quality data.
  arXiv  Detail & Related papers  (2021-07-23T13:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.