MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer

• URL: http://arxiv.org/abs/2309.09067v2
• Date: Tue, 19 Sep 2023 16:24:28 GMT
• Title: MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer
• Authors: Fudong Lin, Summer Crawford, Kaleb Guillot, Yihe Zhang, Yan Chen, Xu Yuan, Li Chen, Shelby Williams, Robert Minvielle, Xiangming Xiao, Drew Gholson, Nicolas Ashwell, Tri Setiyono, Brenda Tubana, Lu Peng, Magdy Bayoumi, Nian-Feng Tzeng
• Abstract summary: We develop a deep learning-based solution, namely Multi-Modal Spatial-Temporal Vision Transformer (MMST-ViT), for predicting crop yields at the county level across the United States. MMST-ViT consists of a Multi-Modal Transformer, a Spatial Transformer, and a Temporal Transformer. We have conducted extensive experiments on over 200 counties in the United States, with the experimental results exhibiting that our MMST-ViT outperforms its counterparts.
• Score: 21.577866420625025
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Precise crop yield prediction provides valuable information for agricultural planning and decision-making processes. However, timely predicting crop yields remains challenging as crop growth is sensitive to growing season weather variation and climate change. In this work, we develop a deep learning-based solution, namely Multi-Modal Spatial-Temporal Vision Transformer (MMST-ViT), for predicting crop yields at the county level across the United States, by considering the effects of short-term meteorological variations during the growing season and the long-term climate change on crops. Specifically, our MMST-ViT consists of a Multi-Modal Transformer, a Spatial Transformer, and a Temporal Transformer. The Multi-Modal Transformer leverages both visual remote sensing data and short-term meteorological data for modeling the effect of growing season weather variations on crop growth. The Spatial Transformer learns the high-resolution spatial dependency among counties for accurate agricultural tracking. The Temporal Transformer captures the long-range temporal dependency for learning the impact of long-term climate change on crops. Meanwhile, we also devise a novel multi-modal contrastive learning technique to pre-train our model without extensive human supervision. Hence, our MMST-ViT captures the impacts of both short-term weather variations and long-term climate change on crops by leveraging both satellite images and meteorological data. We have conducted extensive experiments on over 200 counties in the United States, with the experimental results exhibiting that our MMST-ViT outperforms its counterparts under three performance metrics of interest.

Related papers

• MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling [68.69647625472464]
  Downscaling, a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions. Previous downscaling methods lacked tailored designs for meteorology and encountered structural limitations. We propose a novel model called MambaDS, which enhances the utilization of multivariable correlations and topography information.
  arXiv  Detail & Related papers  (2024-08-20T13:45:49Z)
• VegeDiff: Latent Diffusion Model for Geospatial Vegetation Forecasting [58.12667617617306]
  We propose VegeDiff for the geospatial vegetation forecasting task. VegeDiff is the first to employ a diffusion model to probabilistically capture the uncertainties in vegetation change processes. By capturing the uncertainties in vegetation changes and modeling the complex influence of relevant variables, VegeDiff outperforms existing deterministic methods.
  arXiv  Detail & Related papers  (2024-07-17T14:15:52Z)
• Global Vegetation Modeling with Pre-Trained Weather Transformers [2.238622204691961]
  We adapt the publicly available pre-trained FourCastNet to model vegetation activity. We investigate how the learned global representation of the atmosphere's state can be transferred to model the normalized difference vegetation index (NDVI) We demonstrate that leveraging pre-trained weather models improves the NDVI estimates compared to learning an NDVI model from scratch.
  arXiv  Detail & Related papers  (2024-03-27T10:45:16Z)
• Comparing Data-Driven and Mechanistic Models for Predicting Phenology in Deciduous Broadleaf Forests [47.285748922842444]
  We train a deep neural network to predict a phenological index from meteorological time series. We find that this approach outperforms traditional process-based models.
  arXiv  Detail & Related papers  (2024-01-08T15:29:23Z)
• Inferring the relationship between soil temperature and the normalized difference vegetation index with machine learning [0.3613661942047476]
  Changes in climate can greatly affect the phenology of plants, which can have important feedback effects. In this study, we investigated the effect of soil temperature on the timing of the start of the season. We also explored the impact of other meteorological variables, including air temperature, precipitation, and irradiance, on the inter-annual variation in vegetation phenology.
  arXiv  Detail & Related papers  (2023-12-19T15:43:50Z)
• Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach [71.67506068703314]
  Long-term urban mobility predictions play a crucial role in the effective management of urban facilities and services. Traditionally, urban mobility data has been structured as videos, treating longitude and latitude as fundamental pixels. In our research, we introduce a fresh perspective on urban mobility prediction. Instead of oversimplifying urban mobility data as traditional video data, we regard it as a complex time series.
  arXiv  Detail & Related papers  (2023-12-04T07:39:05Z)
• A novel transformer-based approach for soil temperature prediction [0.0]
  We introduce a novel approach using transformer models for the purpose of forecasting soil temperature prediction. To the best of our knowledge, the usage of transformer models in this work is the very first attempt to predict soil temperature. Experiment results show that the utilization of transformer models ensures a significant contribution to the literature.
  arXiv  Detail & Related papers  (2023-11-20T09:20:26Z)
• Multi-scale Digital Twin: Developing a fast and physics-informed surrogate model for groundwater contamination with uncertain climate models [53.44486283038738]
  Climate change exacerbates the long-term soil management problem of groundwater contamination. We develop a physics-informed machine learning surrogate model using U-Net enhanced Fourier Neural Contaminated (PDENO) In parallel, we develop a convolutional autoencoder combined with climate data to reduce the dimensionality of climatic region similarities across the United States.
  arXiv  Detail & Related papers  (2022-11-20T06:46:35Z)
• Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMs [2.7708222692419735]
  We use a Convolutional LSTM (ConvLSTM) architecture to address this task. We predict changes in the vegetation state in Africa using Sentinel-2 satellite NDVI, having ERA5 weather reanalysis, SMAP satellite measurements, and topography. Ours results highlight how ConvLSTM models can not only forecast the seasonal evolution of NDVI at high resolution, but also the differential impacts of weather anomalies over the baselines.
  arXiv  Detail & Related papers  (2022-10-24T23:03:36Z)
• Dynamical Landscape and Multistability of a Climate Model [64.467612647225]
  We find a third intermediate stable state in one of the two climate models we consider. The combination of our approaches allows to identify how the negative feedback of ocean heat transport and entropy production drastically change the topography of Earth's climate.
  arXiv  Detail & Related papers  (2020-10-20T15:31:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.