CMAViT: Integrating Climate, Managment, and Remote Sensing Data for Crop Yield Estimation with Multimodel Vision Transformers

• URL: http://arxiv.org/abs/2411.16989v1
• Date: Mon, 25 Nov 2024 23:34:53 GMT
• Title: CMAViT: Integrating Climate, Managment, and Remote Sensing Data for Crop Yield Estimation with Multimodel Vision Transformers
• Authors: Hamid Kamangir, Brent. S. Sams, Nick Dokoozlian, Luis Sanchez, J. Mason. Earles,
• Abstract summary: We introduce a deep learning-based multi-model called Climate-Management Aware Vision Transformer (CMAViT) CMAViT integrates both spatial and temporal data by leveraging remote sensing imagery and short-term meteorological data. It outperforms traditional models like UNet-ConvLSTM, excelling in spatial variability capture and yield prediction.
• Score: 0.0
• License:
• Abstract: Crop yield prediction is essential for agricultural planning but remains challenging due to the complex interactions between weather, climate, and management practices. To address these challenges, we introduce a deep learning-based multi-model called Climate-Management Aware Vision Transformer (CMAViT), designed for pixel-level vineyard yield predictions. CMAViT integrates both spatial and temporal data by leveraging remote sensing imagery and short-term meteorological data, capturing the effects of growing season variations. Additionally, it incorporates management practices, which are represented in text form, using a cross-attention encoder to model their interaction with time-series data. This innovative multi-modal transformer tested on a large dataset from 2016-2019 covering 2,200 hectares and eight grape cultivars including more than 5 million vines, outperforms traditional models like UNet-ConvLSTM, excelling in spatial variability capture and yield prediction, particularly for extreme values in vineyards. CMAViT achieved an R2 of 0.84 and a MAPE of 8.22% on an unseen test dataset. Masking specific modalities lowered performance: excluding management practices, climate data, and both reduced R2 to 0.73, 0.70, and 0.72, respectively, and raised MAPE to 11.92%, 12.66%, and 12.39%, highlighting each modality's importance for accurate yield prediction. Code is available at https://github.com/plant-ai-biophysics-lab/CMAViT.

Related papers

• Grapevine Disease Prediction Using Climate Variables from Multi-Sensor Remote Sensing Imagery via a Transformer Model [0.0]
  This paper introduces a novel framework leveraging the TabPFN model to forecast blockwise grapevine diseases. By integrating advanced machine learning techniques with detailed environmental data, our approach significantly enhances the accuracy and efficiency of disease prediction.
  arXiv  Detail & Related papers  (2024-06-11T09:33:15Z)
• An Open and Large-Scale Dataset for Multi-Modal Climate Change-aware Crop Yield Predictions [20.44172558372343]
  CropNet dataset is the first terabyte-sized, publicly available, and multi-modal dataset specifically targeting climate change-aware crop yield predictions. CropNet dataset is composed of three modalities of data, i.e., Sentinel-2 Imagery, WRF-HRRR, and USDA Cropd dataset, for over 2200 U.S. counties spanning 6 years.
  arXiv  Detail & Related papers  (2024-06-10T07:54:56Z)
• Adaptive Fusion of Multi-view Remote Sensing data for Optimal Sub-field Crop Yield Prediction [24.995959334158986]
  We present a novel multi-view learning approach to predict crop yield for different crops (soybean, wheat, rapeseed) and regions (Argentina, Uruguay, and Germany). Our input data includes multi-spectral optical images from Sentinel-2 satellites and weather data as dynamic features during the crop growing season, complemented by static features like soil properties and topographic information. To effectively fuse the data, we introduce a Multi-view Gated Fusion (MVGF) model, comprising dedicated view-encoders and a Gated Unit (GU) module. The MVGF model is trained at sub-field level with 10 m resolution
  arXiv  Detail & Related papers  (2024-01-22T11:01:52Z)
• A Hybrid Deep Learning-based Approach for Optimal Genotype by Environment Selection [8.084449311613517]
  We used a dataset comprising 93,028 training records to forecast yields for 10,337 test records, covering 159 locations over 13 years (2003-2015) This dataset included details on 5,838 distinct genotypes and daily weather data for a 214-day growing season, enabling comprehensive analysis. We developed two novel convolutional neural network (CNN) architectures: the CNN-DNN model, combining CNN and fully-connected networks, and the CNN-LSTM-DNN model, with an added LSTM layer for weather variables.
  arXiv  Detail & Related papers  (2023-09-22T17:31:47Z)
• Vision Transformers, a new approach for high-resolution and large-scale mapping of canopy heights [50.52704854147297]
  We present a new vision transformer (ViT) model optimized with a classification (discrete) and a continuous loss function. This model achieves better accuracy than previously used convolutional based approaches (ConvNets) optimized with only a continuous loss function.
  arXiv  Detail & Related papers  (2023-04-22T22:39:03Z)
• Generative models-based data labeling for deep networks regression: application to seed maturity estimation from UAV multispectral images [3.6868861317674524]
  Monitoring seed maturity is an increasing challenge in agriculture due to climate change and more restrictive practices. Traditional methods are based on limited sampling in the field and analysis in laboratory. We propose a method for estimating parsley seed maturity using multispectral UAV imagery, with a new approach for automatic data labeling.
  arXiv  Detail & Related papers  (2022-08-09T09:06:51Z)
• End-to-end deep learning for directly estimating grape yield from ground-based imagery [53.086864957064876]
  This study demonstrates the application of proximal imaging combined with deep learning for yield estimation in vineyards. Three model architectures were tested: object detection, CNN regression, and transformer models. The study showed the applicability of proximal imaging and deep learning for prediction of grapevine yield on a large scale.
  arXiv  Detail & Related papers  (2022-08-04T01:34:46Z)
• Global Context Vision Transformers [78.5346173956383]
  We propose global context vision transformer (GC ViT), a novel architecture that enhances parameter and compute utilization for computer vision. We address the lack of the inductive bias in ViTs, and propose to leverage a modified fused inverted residual blocks in our architecture. Our proposed GC ViT achieves state-of-the-art results across image classification, object detection and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2022-06-20T18:42:44Z)
• MSeg: A Composite Dataset for Multi-domain Semantic Segmentation [100.17755160696939]
  We present MSeg, a composite dataset that unifies semantic segmentation datasets from different domains. We reconcile the generalization and bring the pixel-level annotations into alignment by relabeling more than 220,000 object masks in more than 80,000 images. A model trained on MSeg ranks first on the WildDash-v1 leaderboard for robust semantic segmentation, with no exposure to WildDash data during training.
  arXiv  Detail & Related papers  (2021-12-27T16:16:35Z)
• A CNN Approach to Simultaneously Count Plants and Detect Plantation-Rows from UAV Imagery [56.10033255997329]
  We propose a novel deep learning method based on a Convolutional Neural Network (CNN) It simultaneously detects and geolocates plantation-rows while counting its plants considering highly-dense plantation configurations. The proposed method achieved state-of-the-art performance for counting and geolocating plants and plant-rows in UAV images from different types of crops.
  arXiv  Detail & Related papers  (2020-12-31T18:51:17Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.