A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition

• URL: http://arxiv.org/abs/2309.13578v1
• Date: Sun, 24 Sep 2023 08:34:12 GMT
• Title: A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition
• Authors: Khoa Dang Nguyen, Thanh-Hai Phung, Hoang-Giang Cao
• Abstract summary: We propose an approach that combines the effectiveness of the Segment AnyThing Model (SAM) for instance segmentation with prompt input from object detection models. Our best-performing model achieved a PQ+ score of 81.33 based on the evaluation metrics of the competition.
• Score: 2.7624021966289605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Panoptic segmentation in agriculture is an advanced computer vision technique that provides a comprehensive understanding of field composition. It facilitates various tasks such as crop and weed segmentation, plant panoptic segmentation, and leaf instance segmentation, all aimed at addressing challenges in agriculture. Exploring the application of panoptic segmentation in agriculture, the 8th Workshop on Computer Vision in Plant Phenotyping and Agriculture (CVPPA) hosted the challenge of hierarchical panoptic segmentation of crops and weeds using the PhenoBench dataset. To tackle the tasks presented in this competition, we propose an approach that combines the effectiveness of the Segment AnyThing Model (SAM) for instance segmentation with prompt input from object detection models. Specifically, we integrated two notable approaches in object detection, namely DINO and YOLO-v8. Our best-performing model achieved a PQ+ score of 81.33 based on the evaluation metrics of the competition.

Related papers

• Investigating the Segment Anything Foundation Model for Mapping Smallholder Agriculture Field Boundaries Without Training Labels [0.24966046892475396]
  This study explores the Segment Anything Model (SAM) to delineate agricultural field boundaries in Bihar, India. We evaluate SAM's performance across three model checkpoints, various input sizes, multi-date satellite images, and edge-enhanced imagery. Using different input image sizes improves accuracy, with the most significant improvement observed when using multi-date satellite images.
  arXiv  Detail & Related papers  (2024-07-01T23:06:02Z)
• Agriculture-Vision Challenge 2024 -- The Runner-Up Solution for Agricultural Pattern Recognition via Class Balancing and Model Ensemble [20.631333392618327]
  The Agriculture-Vision Challenge at CVPR 2024 aims at leveraging semantic segmentation models to produce pixel level semantic segmentation labels. We propose a mosaic data augmentation with a rare class sampling strategy to enrich long-tail class samples. Our methodology achieved a mean Intersection over Union (mIoU) score of 0.547 on the test set, securing second place in this challenge.
  arXiv  Detail & Related papers  (2024-06-18T04:59:04Z)
• Generating Diverse Agricultural Data for Vision-Based Farming Applications [74.79409721178489]
  This model is capable of simulating distinct growth stages of plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture.
  arXiv  Detail & Related papers  (2024-03-27T08:42:47Z)
• Can SAM recognize crops? Quantifying the zero-shot performance of a semantic segmentation foundation model on generating crop-type maps using satellite imagery for precision agriculture [4.825257766966091]
  Crop-type maps are key information for decision-support tools. We investigate the capabilities of Meta AI's Segment Anything Model (SAM) for crop-map prediction task. SAM being limited to up-to 3 channel inputs and its zero-shot usage being class-agnostic in nature pose unique challenges in using it directly for crop-type mapping.
  arXiv  Detail & Related papers  (2023-11-25T23:40:09Z)
• AIMS: All-Inclusive Multi-Level Segmentation [93.5041381700744]
  We propose a new task, All-Inclusive Multi-Level (AIMS), which segments visual regions into three levels: part, entity, and relation. We also build a unified AIMS model through multi-dataset multi-task training to address the two major challenges of annotation inconsistency and task correlation.
  arXiv  Detail & Related papers  (2023-05-28T16:28:49Z)
• Semantic Image Segmentation with Deep Learning for Vine Leaf Phenotyping [59.0626764544669]
  In this study, we use Deep Learning methods to semantically segment grapevine leaves images in order to develop an automated object detection system for leaf phenotyping. Our work contributes to plant lifecycle monitoring through which dynamic traits such as growth and development can be captured and quantified.
  arXiv  Detail & Related papers  (2022-10-24T14:37:09Z)
• Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain [29.647846446064992]
  Plant phenotyping is a central task in agriculture, as it describes plants' growth stage, development, and other relevant quantities. In this paper, we address the problem of joint semantic, plant instance, and leaf instance segmentation of crop fields from RGB data. We propose a single convolutional neural network that addresses the three tasks simultaneously, exploiting their underlying hierarchical structure.
  arXiv  Detail & Related papers  (2022-10-14T15:01:08Z)
• Pollen13K: A Large Scale Microscope Pollen Grain Image Dataset [63.05335933454068]
  This work presents the first large-scale pollen grain image dataset, including more than 13 thousands objects. The paper focuses on the employed data acquisition steps, which include aerobiological sampling, microscope image acquisition, object detection, segmentation and labelling.
  arXiv  Detail & Related papers  (2020-07-09T10:33:31Z)
• The 1st Agriculture-Vision Challenge: Methods and Results [144.57794061346974]
  The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images. Around 57 participating teams from various countries compete to achieve state-of-the-art in aerial agriculture semantic segmentation. This paper provides a summary of notable methods and results in the challenge.
  arXiv  Detail & Related papers  (2020-04-21T05:02:31Z)
• Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis [110.30849704592592]
  We present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns. Each image consists of RGB and Near-infrared (NIR) channels with resolution as high as 10 cm per pixel. We annotate nine types of field anomaly patterns that are most important to farmers.
  arXiv  Detail & Related papers  (2020-01-05T20:19:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.