Multi-modal learning for geospatial vegetation forecasting

• URL: http://arxiv.org/abs/2303.16198v2
• Date: Thu, 7 Mar 2024 14:42:09 GMT
• Title: Multi-modal learning for geospatial vegetation forecasting
• Authors: Vitus Benson, Claire Robin, Christian Requena-Mesa, Lazaro Alonso, Nuno Carvalhais, Jos\'e Cort\'es, Zhihan Gao, Nora Linscheid, M\'elanie Weynants, Markus Reichstein
• Abstract summary: We introduce GreenEarthNet, the first dataset specifically designed for high-resolution vegetation forecasting. We also present Contextformer, a novel deep learning approach for predicting vegetation greenness from Sentinel 2 satellite images. To the best of our knowledge, this work presents the first models for continental-scale vegetation modeling at fine resolution able to capture anomalies beyond the seasonal cycle.
• Score: 1.8180482634934092
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The innovative application of precise geospatial vegetation forecasting holds immense potential across diverse sectors, including agriculture, forestry, humanitarian aid, and carbon accounting. To leverage the vast availability of satellite imagery for this task, various works have applied deep neural networks for predicting multispectral images in photorealistic quality. However, the important area of vegetation dynamics has not been thoroughly explored. Our study breaks new ground by introducing GreenEarthNet, the first dataset specifically designed for high-resolution vegetation forecasting, and Contextformer, a novel deep learning approach for predicting vegetation greenness from Sentinel 2 satellite images with fine resolution across Europe. Our multi-modal transformer model Contextformer leverages spatial context through a vision backbone and predicts the temporal dynamics on local context patches incorporating meteorological time series in a parameter-efficient manner. The GreenEarthNet dataset features a learned cloud mask and an appropriate evaluation scheme for vegetation modeling. It also maintains compatibility with the existing satellite imagery forecasting dataset EarthNet2021, enabling cross-dataset model comparisons. Our extensive qualitative and quantitative analyses reveal that our methods outperform a broad range of baseline techniques. This includes surpassing previous state-of-the-art models on EarthNet2021, as well as adapted models from time series forecasting and video prediction. To the best of our knowledge, this work presents the first models for continental-scale vegetation modeling at fine resolution able to capture anomalies beyond the seasonal cycle, thereby paving the way for predicting vegetation health and behaviour in response to climate variability and extremes.

Related papers

• MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling [68.69647625472464]
  Downscaling, a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions. Previous downscaling methods lacked tailored designs for meteorology and encountered structural limitations. We propose a novel model called MambaDS, which enhances the utilization of multivariable correlations and topography information.
  arXiv  Detail & Related papers  (2024-08-20T13:45:49Z)
• Towards a Knowledge guided Multimodal Foundation Model for Spatio-Temporal Remote Sensing Applications [16.824262496666893]
  We present a foundation model framework, where the pretraining task captures the causal relationship between multiple modalities. Our method, called MultiModal Variable Step Forecasting (MM-VSF), uses forecasting of satellite imagery as a pretraining task.
  arXiv  Detail & Related papers  (2024-07-29T02:49:55Z)
• Global Vegetation Modeling with Pre-Trained Weather Transformers [2.238622204691961]
  We adapt the publicly available pre-trained FourCastNet to model vegetation activity. We investigate how the learned global representation of the atmosphere's state can be transferred to model the normalized difference vegetation index (NDVI) We demonstrate that leveraging pre-trained weather models improves the NDVI estimates compared to learning an NDVI model from scratch.
  arXiv  Detail & Related papers  (2024-03-27T10:45:16Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• Comparing Data-Driven and Mechanistic Models for Predicting Phenology in Deciduous Broadleaf Forests [47.285748922842444]
  We train a deep neural network to predict a phenological index from meteorological time series. We find that this approach outperforms traditional process-based models.
  arXiv  Detail & Related papers  (2024-01-08T15:29:23Z)
• Deep autoregressive modeling for land use land cover [0.0]
  Land use / land cover (LULC) modeling is a challenging task due to long-range dependencies between geographic features and distinct spatial patterns related to topography, ecology, and human development. We identify a close connection between modeling of spatial patterns of land use and the task of image inpainting from computer vision and conduct a study of a modified PixelCNN architecture with approximately 19 million parameters for modeling LULC.
  arXiv  Detail & Related papers  (2024-01-02T18:03:57Z)
• Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMs [2.7708222692419735]
  We use a Convolutional LSTM (ConvLSTM) architecture to address this task. We predict changes in the vegetation state in Africa using Sentinel-2 satellite NDVI, having ERA5 weather reanalysis, SMAP satellite measurements, and topography. Ours results highlight how ConvLSTM models can not only forecast the seasonal evolution of NDVI at high resolution, but also the differential impacts of weather anomalies over the baselines.
  arXiv  Detail & Related papers  (2022-10-24T23:03:36Z)
• Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
  We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments. Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion. We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
  arXiv  Detail & Related papers  (2022-06-29T07:49:48Z)
• Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark [95.19070157520633]
  Recent progress in self-supervision shows that pre-training large neural networks on vast amounts of unsupervised data can lead to impressive increases in generalisation for downstream tasks. Such models, recently coined as foundation models, have been transformational to the field of natural language processing. We propose to develop a new benchmark comprised of a variety of downstream tasks related to climate change.
  arXiv  Detail & Related papers  (2021-12-01T15:38:19Z)
• EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts [12.795776149170978]
  Large Earth observation datasets now enable us to create machine learning models capable of translating coarse weather information into high-resolution Earth surface forecasts. We define high-resolution Earth surface forecasting as video prediction of satellite imagery conditional on mesoscale weather forecasts. We introduce EarthNet 2021, a new curated dataset containing target-temporal Sentinel 2 satellite imagery at 20 m resolution, matched with high-resolution topography and mesoscale (1.28 km) weather variables.
  arXiv  Detail & Related papers  (2020-12-11T11:21:00Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.