Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Point Clouds

• URL: http://arxiv.org/abs/2411.07799v1
• Date: Tue, 12 Nov 2024 13:53:22 GMT
• Title: Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Point Clouds
• Authors: Daniel Fusaro, Federico Magistri, Jens Behley, Alberto Pretto, Cyrill Stachniss,
• Abstract summary: We present a novel approach for temporal fruit monitoring that addresses point clouds collected in a greenhouse over time. Our method segments fruits using a learning-based instance segmentation approach directly on the point cloud. Experimental results on a real dataset of strawberries demonstrate that our approach outperforms other methods for fruits re-identification over time.
• Score: 29.23207854514898
• License:
• Abstract: Robotic fruit monitoring is a key step toward automated agricultural production systems. Robots can significantly enhance plant and temporal fruit monitoring by providing precise, high-throughput assessments that overcome the limitations of traditional manual methods. Fruit monitoring is a challenging task due to the significant variation in size, shape, orientation, and occlusion of fruits. Also, fruits may be harvested or newly grown between recording sessions. Most methods are 2D image-based and they lack the 3D structure, depth, and spatial information, which represent key aspects of fruit monitoring. 3D colored point clouds, instead, can offer this information but they introduce challenges such as their sparsity and irregularity. In this paper, we present a novel approach for temporal fruit monitoring that addresses point clouds collected in a greenhouse over time. Our method segments fruits using a learning-based instance segmentation approach directly on the point cloud. Each segmented fruit is processed by a 3D sparse convolutional neural network to extract descriptors, which are used in an attention-based matching network to associate fruits with their instances from previous data collections. Experimental results on a real dataset of strawberries demonstrate that our approach outperforms other methods for fruits re-identification over time, allowing for precise temporal fruit monitoring in real and complex scenarios.

Related papers

• FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework [5.363729942767801]
  We introduce FruitNeRF, a unified novel fruit counting framework. We use state-of-the-art view synthesis methods to count any fruit type directly in 3D. We evaluate our methodology using both real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2024-08-12T14:40:38Z)
• Few-Shot Fruit Segmentation via Transfer Learning [4.616529139444651]
  We develop a few-shot semantic segmentation framework for infield fruits using transfer learning. Motivated by similar success in urban scene parsing, we propose specialized pre-training. We show that models with pre-training learn to distinguish between fruit still on the trees and fruit that have fallen on the ground.
  arXiv  Detail & Related papers  (2024-05-04T04:05:59Z)
• A pipeline for multiple orange detection and tracking with 3-D fruit relocalization and neural-net based yield regression in commercial citrus orchards [0.0]
  We propose a non-invasive alternative that utilizes fruit counting from videos, implemented as a pipeline. To handle occluded and re-appeared fruit, we introduce a relocalization component that employs 3-D estimation of fruit locations. By ensuring that at least 30% of the fruit is accurately detected, tracked, and counted, our yield regressor achieves an impressive coefficient of determination of 0.85.
  arXiv  Detail & Related papers  (2023-12-27T21:22:43Z)
• Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots [33.21287030243106]
  Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. We address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot.
  arXiv  Detail & Related papers  (2023-03-15T20:41:24Z)
• Fruit Ripeness Classification: a Survey [59.11160990637616]
  Many automatic methods have been proposed that employ a variety of feature descriptors for the food item to be graded. Machine learning and deep learning techniques dominate the top-performing methods. Deep learning can operate on raw data and thus relieve the users from having to compute complex engineered features.
  arXiv  Detail & Related papers  (2022-12-29T19:32:20Z)
• Semantic Image Segmentation with Deep Learning for Vine Leaf Phenotyping [59.0626764544669]
  In this study, we use Deep Learning methods to semantically segment grapevine leaves images in order to develop an automated object detection system for leaf phenotyping. Our work contributes to plant lifecycle monitoring through which dynamic traits such as growth and development can be captured and quantified.
  arXiv  Detail & Related papers  (2022-10-24T14:37:09Z)
• Facilitated machine learning for image-based fruit quality assessment in developing countries [68.8204255655161]
  Automated image classification is a common task for supervised machine learning in food science. We propose an alternative method based on pre-trained vision transformers (ViTs) It can be easily implemented with limited resources on a standard device.
  arXiv  Detail & Related papers  (2022-07-10T19:52:20Z)
• A methodology for detection and localization of fruits in apples orchards from aerial images [0.0]
  This work presents a methodology for automated fruit counting employing aerial-images. It includes algorithms based on multiple view geometry to perform fruits tracking. Preliminary assessments show correlations above 0.8 between fruit counting and true yield for apples.
  arXiv  Detail & Related papers  (2021-10-24T01:57:52Z)
• Point Discriminative Learning for Unsupervised Representation Learning on 3D Point Clouds [54.31515001741987]
  We propose a point discriminative learning method for unsupervised representation learning on 3D point clouds. We achieve this by imposing a novel point discrimination loss on the middle level and global level point features. Our method learns powerful representations and achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-08-04T15:11:48Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.