A novel transformer-based approach for soil temperature prediction

• URL: http://arxiv.org/abs/2311.11626v1
• Date: Mon, 20 Nov 2023 09:20:26 GMT
• Title: A novel transformer-based approach for soil temperature prediction
• Authors: Muhammet Mucahit Enes Yurtsever, Ayhan Kucukmanisa and Zeynep Hilal Kilimci
• Abstract summary: We introduce a novel approach using transformer models for the purpose of forecasting soil temperature prediction. To the best of our knowledge, the usage of transformer models in this work is the very first attempt to predict soil temperature. Experiment results show that the utilization of transformer models ensures a significant contribution to the literature.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Soil temperature is one of the most significant parameters that plays a crucial role in glacier energy, dynamics of mass balance, processes of surface hydrological, coaction of glacier-atmosphere, nutrient cycling, ecological stability, the management of soil, water, and field crop. In this work, we introduce a novel approach using transformer models for the purpose of forecasting soil temperature prediction. To the best of our knowledge, the usage of transformer models in this work is the very first attempt to predict soil temperature. Experiments are carried out using six different FLUXNET stations by modeling them with five different transformer models, namely, Vanilla Transformer, Informer, Autoformer, Reformer, and ETSformer. To demonstrate the effectiveness of the proposed model, experiment results are compared with both deep learning approaches and literature studies. Experiment results show that the utilization of transformer models ensures a significant contribution to the literature, thence determining the new state-of-the-art.

Related papers

• VegeDiff: Latent Diffusion Model for Geospatial Vegetation Forecasting [58.12667617617306]
  We propose VegeDiff for the geospatial vegetation forecasting task. VegeDiff is the first to employ a diffusion model to probabilistically capture the uncertainties in vegetation change processes. By capturing the uncertainties in vegetation changes and modeling the complex influence of relevant variables, VegeDiff outperforms existing deterministic methods.
  arXiv  Detail & Related papers  (2024-07-17T14:15:52Z)
• Embedding machine-learnt sub-grid variability improves climate model biases [0.44998333629984877]
  Under-representation of cloud formation is a long-standing bias associated with climate simulations. We overcome these biases by embedding a Multi-Output Gaussian Process (MOGP) trained on high resolution Unified Model simulations. Ten-year predictions are generated for both control and ML-hybrid models.
  arXiv  Detail & Related papers  (2024-06-13T19:35:58Z)
• Decomposing weather forecasting into advection and convection with neural networks [6.78786601630176]
  We propose a simple yet effective machine learning model that learns the horizontal movement in the dynamical core and vertical movement in the physical parameterization separately. Our model provides a new and efficient perspective to simulate the transition of variables in atmospheric models.
  arXiv  Detail & Related papers  (2024-05-10T16:46:32Z)
• Global Vegetation Modeling with Pre-Trained Weather Transformers [2.238622204691961]
  We adapt the publicly available pre-trained FourCastNet to model vegetation activity. We investigate how the learned global representation of the atmosphere's state can be transferred to model the normalized difference vegetation index (NDVI) We demonstrate that leveraging pre-trained weather models improves the NDVI estimates compared to learning an NDVI model from scratch.
  arXiv  Detail & Related papers  (2024-03-27T10:45:16Z)
• Comparing Data-Driven and Mechanistic Models for Predicting Phenology in Deciduous Broadleaf Forests [47.285748922842444]
  We train a deep neural network to predict a phenological index from meteorological time series. We find that this approach outperforms traditional process-based models.
  arXiv  Detail & Related papers  (2024-01-08T15:29:23Z)
• MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer [21.577866420625025]
  We develop a deep learning-based solution, namely Multi-Modal Spatial-Temporal Vision Transformer (MMST-ViT), for predicting crop yields at the county level across the United States. MMST-ViT consists of a Multi-Modal Transformer, a Spatial Transformer, and a Temporal Transformer. We have conducted extensive experiments on over 200 counties in the United States, with the experimental results exhibiting that our MMST-ViT outperforms its counterparts.
  arXiv  Detail & Related papers  (2023-09-16T18:22:20Z)
• A Three-regime Model of Network Pruning [47.92525418773768]
  We use temperature-like and load-like parameters to model the impact of neural network (NN) training hyper parameters on pruning performance. A key empirical result we identify is a sharp transition phenomenon: depending on the value of a load-like parameter in the pruned model, increasing the value of a temperature-like parameter in the pre-pruned model may either enhance or impair subsequent pruning performance. Our model reveals that the dichotomous effect of high temperature is associated with transitions between distinct types of global structures in the post-pruned model.
  arXiv  Detail & Related papers  (2023-05-28T08:09:25Z)
• A Comparative Study on Generative Models for High Resolution Solar Observation Imaging [59.372588316558826]
  This work investigates capabilities of current state-of-the-art generative models to accurately capture the data distribution behind observed solar activity states. Using distributed training on supercomputers, we are able to train generative models for up to 1024x1024 resolution that produce high quality samples indistinguishable to human experts.
  arXiv  Detail & Related papers  (2023-04-14T14:40:32Z)
• Measuring Wind Turbine Health Using Drifting Concepts [55.87342698167776]
  We propose two new approaches for the analysis of wind turbine health. The first method aims at evaluating the decrease or increase in relatively high and low power production. The second method evaluates the overall drift of the extracted concepts.
  arXiv  Detail & Related papers  (2021-12-09T14:04:55Z)
• Physics-Guided Generative Adversarial Networks for Sea Subsurface Temperature Prediction [24.55780949103687]
  Sea subsurface temperature is affected by global warming in climate change. Existing research is commonly based on either physics-based numerical models or data based models. We propose a novel framework based on generative adversarial network (GAN) combined with numerical model to predict sea subsurface temperature.
  arXiv  Detail & Related papers  (2021-11-04T23:46:51Z)
• Dynamical Landscape and Multistability of a Climate Model [64.467612647225]
  We find a third intermediate stable state in one of the two climate models we consider. The combination of our approaches allows to identify how the negative feedback of ocean heat transport and entropy production drastically change the topography of Earth's climate.
  arXiv  Detail & Related papers  (2020-10-20T15:31:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.