Can SAM recognize crops? Quantifying the zero-shot performance of a semantic segmentation foundation model on generating crop-type maps using satellite imagery for precision agriculture

• URL: http://arxiv.org/abs/2311.15138v2
• Date: Mon, 4 Dec 2023 21:02:05 GMT
• Title: Can SAM recognize crops? Quantifying the zero-shot performance of a semantic segmentation foundation model on generating crop-type maps using satellite imagery for precision agriculture
• Authors: Rutuja Gurav, Het Patel, Zhuocheng Shang, Ahmed Eldawy, Jia Chen, Elia Scudiero, Evangelos Papalexakis
• Abstract summary: Crop-type maps are key information for decision-support tools. We investigate the capabilities of Meta AI's Segment Anything Model (SAM) for crop-map prediction task. SAM being limited to up-to 3 channel inputs and its zero-shot usage being class-agnostic in nature pose unique challenges in using it directly for crop-type mapping.
• Score: 4.825257766966091
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Climate change is increasingly disrupting worldwide agriculture, making global food production less reliable. To tackle the growing challenges in feeding the planet, cutting-edge management strategies, such as precision agriculture, empower farmers and decision-makers with rich and actionable information to increase the efficiency and sustainability of their farming practices. Crop-type maps are key information for decision-support tools but are challenging and costly to generate. We investigate the capabilities of Meta AI's Segment Anything Model (SAM) for crop-map prediction task, acknowledging its recent successes at zero-shot image segmentation. However, SAM being limited to up-to 3 channel inputs and its zero-shot usage being class-agnostic in nature pose unique challenges in using it directly for crop-type mapping. We propose using clustering consensus metrics to assess SAM's zero-shot performance in segmenting satellite imagery and producing crop-type maps. Although direct crop-type mapping is challenging using SAM in zero-shot setting, experiments reveal SAM's potential for swiftly and accurately outlining fields in satellite images, serving as a foundation for subsequent crop classification. This paper attempts to highlight a use-case of state-of-the-art image segmentation models like SAM for crop-type mapping and related specific needs of the agriculture industry, offering a potential avenue for automatic, efficient, and cost-effective data products for precision agriculture practices.

Related papers

• Investigating the Segment Anything Foundation Model for Mapping Smallholder Agriculture Field Boundaries Without Training Labels [0.24966046892475396]
  This study explores the Segment Anything Model (SAM) to delineate agricultural field boundaries in Bihar, India. We evaluate SAM's performance across three model checkpoints, various input sizes, multi-date satellite images, and edge-enhanced imagery. Using different input image sizes improves accuracy, with the most significant improvement observed when using multi-date satellite images.
  arXiv  Detail & Related papers  (2024-07-01T23:06:02Z)
• Generating Diverse Agricultural Data for Vision-Based Farming Applications [74.79409721178489]
  This model is capable of simulating distinct growth stages of plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture.
  arXiv  Detail & Related papers  (2024-03-27T08:42:47Z)
• Combining Satellite and Weather Data for Crop Type Mapping: An Inverse Modelling Approach [23.23933321161625]
  We propose a deep learning model that combines weather (Daymet) and satellite imagery (Sentinel-2) to generate accurate crop maps. We show that our approach provides significant improvements over existing algorithms that solely rely on spectral imagery. We conclude by correlating our results with crop phenology to show that WSTATT is able to capture physical properties of crop growth.
  arXiv  Detail & Related papers  (2024-01-29T04:15:22Z)
• AI for Agriculture: the Comparison of Semantic Segmentation Methods for Crop Mapping with Sentinel-2 Imagery [0.0]
  Crop mapping is one of the most common tasks in artificial intelligence for agriculture. With higher resolution satellite imagery the texture is easily detected by majority of state-of-the-art algorithms. In this paper we aim to explore the main machine learning methods that can be used with freely available satellite imagery.
  arXiv  Detail & Related papers  (2023-11-21T21:00:42Z)
• A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition [2.7624021966289605]
  We propose an approach that combines the effectiveness of the Segment AnyThing Model (SAM) for instance segmentation with prompt input from object detection models. Our best-performing model achieved a PQ+ score of 81.33 based on the evaluation metrics of the competition.
  arXiv  Detail & Related papers  (2023-09-24T08:34:12Z)
• HarvestNet: A Dataset for Detecting Smallholder Farming Activity Using Harvest Piles and Remote Sensing [50.4506590177605]
  HarvestNet is a dataset for mapping the presence of farms in the Ethiopian regions of Tigray and Amhara during 2020-2023. We introduce a new approach based on the detection of harvest piles characteristic of many smallholder systems. We conclude that remote sensing of harvest piles can contribute to more timely and accurate cropland assessments in food insecure regions.
  arXiv  Detail & Related papers  (2023-08-23T11:03:28Z)
• Agave crop segmentation and maturity classification with deep learning data-centric strategies using very high-resolution satellite imagery [101.18253437732933]
  We present an Agave tequilana Weber azul crop segmentation and maturity classification using very high resolution satellite imagery. We solve real-world deep learning problems in the very specific context of agave crop segmentation. With the resulting accurate models, agave production forecasting can be made available for large regions.
  arXiv  Detail & Related papers  (2023-03-21T03:15:29Z)
• Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection [60.83360138070649]
  The paper presents an approach for analyzing aerial images of a potato crop using deep neural networks. The main objective is to demonstrate automated spatial recognition of a healthy versus stressed crop at a plant level. Experimental validation demonstrated the ability for distinguishing healthy and stressed plants in field images, achieving an average Dice coefficient of 0.74.
  arXiv  Detail & Related papers  (2021-06-14T21:57:40Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)
• UAV and Machine Learning Based Refinement of a Satellite-Driven Vegetation Index for Precision Agriculture [0.8399688944263843]
  This paper presents a novel satellite imagery refinement framework based on a deep learning technique. It exploits information properly derived from high resolution images acquired by unmanned aerial vehicle (UAV) airborne multispectral sensors. A vineyard in Serralunga d'Alba (Northern Italy) was chosen as a case study for validation purposes.
  arXiv  Detail & Related papers  (2020-04-29T18:34:48Z)
• Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis [110.30849704592592]
  We present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns. Each image consists of RGB and Near-infrared (NIR) channels with resolution as high as 10 cm per pixel. We annotate nine types of field anomaly patterns that are most important to farmers.
  arXiv  Detail & Related papers  (2020-01-05T20:19:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.