Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Colored Point Clouds

• URL: http://arxiv.org/abs/2411.07799v2
• Date: Fri, 18 Jul 2025 11:52:19 GMT
• Title: Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Colored Point Clouds
• Authors: Daniel Fusaro, Federico Magistri, Jens Behley, Alberto Pretto, Cyrill Stachniss,
• Abstract summary: We propose a novel method for fruit instance segmentation and re-identification on 3D terrestrial point clouds collected over time.<n>Our approach directly operates on dense colored point clouds, capturing fine-grained 3D spatial detail.<n>We evaluate our approach on real-world datasets of strawberries and apples, demonstrating that it outperforms existing methods in both instance segmentation and temporal re-identification.
• Score: 29.23207854514898
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate and consistent fruit monitoring over time is a key step toward automated agricultural production systems. However, this task is inherently difficult due to variations in fruit size, shape, occlusion, orientation, and the dynamic nature of orchards where fruits may appear or disappear between observations. In this article, we propose a novel method for fruit instance segmentation and re-identification on 3D terrestrial point clouds collected over time. Our approach directly operates on dense colored point clouds, capturing fine-grained 3D spatial detail. We segment individual fruits using a learning-based instance segmentation method applied directly to the point cloud. For each segmented fruit, we extract a compact and discriminative descriptor using a 3D sparse convolutional neural network. To track fruits across different times, we introduce an attention-based matching network that associates fruits with their counterparts from previous sessions. Matching is performed using a probabilistic assignment scheme, selecting the most likely associations across time. We evaluate our approach on real-world datasets of strawberries and apples, demonstrating that it outperforms existing methods in both instance segmentation and temporal re-identification, enabling robust and precise fruit monitoring across complex and dynamic orchard environments.

Related papers

• FruitNeRF++: A Generalized Multi-Fruit Counting Method Utilizing Contrastive Learning and Neural Radiance Fields [5.669562773066833]
  We introduce FruitNeRF++, a novel fruit-counting approach that combines contrastive learning with neural radiance fields.<n>Our work is based on FruitNeRF, which employs a neural semantic field combined with a fruit-specific clustering approach.<n>Our results demonstrate that FruitNeRF++ is easier to control and compares favorably to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-26T11:48:22Z)
• FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework [5.363729942767801]
  We introduce FruitNeRF, a unified novel fruit counting framework. We use state-of-the-art view synthesis methods to count any fruit type directly in 3D. We evaluate our methodology using both real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2024-08-12T14:40:38Z)
• View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
  We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene. Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output. We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
  arXiv  Detail & Related papers  (2024-05-30T04:14:58Z)
• Few-Shot Fruit Segmentation via Transfer Learning [4.616529139444651]
  We develop a few-shot semantic segmentation framework for infield fruits using transfer learning. Motivated by similar success in urban scene parsing, we propose specialized pre-training. We show that models with pre-training learn to distinguish between fruit still on the trees and fruit that have fallen on the ground.
  arXiv  Detail & Related papers  (2024-05-04T04:05:59Z)
• A pipeline for multiple orange detection and tracking with 3-D fruit relocalization and neural-net based yield regression in commercial citrus orchards [0.0]
  We propose a non-invasive alternative that utilizes fruit counting from videos, implemented as a pipeline. To handle occluded and re-appeared fruit, we introduce a relocalization component that employs 3-D estimation of fruit locations. By ensuring that at least 30% of the fruit is accurately detected, tracked, and counted, our yield regressor achieves an impressive coefficient of determination of 0.85.
  arXiv  Detail & Related papers  (2023-12-27T21:22:43Z)
• Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots [33.21287030243106]
  Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. We address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot.
  arXiv  Detail & Related papers  (2023-03-15T20:41:24Z)
• Fruit Ripeness Classification: a Survey [59.11160990637616]
  Many automatic methods have been proposed that employ a variety of feature descriptors for the food item to be graded. Machine learning and deep learning techniques dominate the top-performing methods. Deep learning can operate on raw data and thus relieve the users from having to compute complex engineered features.
  arXiv  Detail & Related papers  (2022-12-29T19:32:20Z)
• Semantic Image Segmentation with Deep Learning for Vine Leaf Phenotyping [59.0626764544669]
  In this study, we use Deep Learning methods to semantically segment grapevine leaves images in order to develop an automated object detection system for leaf phenotyping. Our work contributes to plant lifecycle monitoring through which dynamic traits such as growth and development can be captured and quantified.
  arXiv  Detail & Related papers  (2022-10-24T14:37:09Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• Facilitated machine learning for image-based fruit quality assessment in developing countries [68.8204255655161]
  Automated image classification is a common task for supervised machine learning in food science. We propose an alternative method based on pre-trained vision transformers (ViTs) It can be easily implemented with limited resources on a standard device.
  arXiv  Detail & Related papers  (2022-07-10T19:52:20Z)
• A methodology for detection and localization of fruits in apples orchards from aerial images [0.0]
  This work presents a methodology for automated fruit counting employing aerial-images. It includes algorithms based on multiple view geometry to perform fruits tracking. Preliminary assessments show correlations above 0.8 between fruit counting and true yield for apples.
  arXiv  Detail & Related papers  (2021-10-24T01:57:52Z)
• Unsupervised Representation Learning for 3D Point Cloud Data [66.92077180228634]
  We propose a simple yet effective approach for unsupervised point cloud learning. In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud. We conduct experiments on three downstream tasks which are 3D object classification, shape part segmentation and scene segmentation.
  arXiv  Detail & Related papers  (2021-10-13T10:52:45Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)
• Point Discriminative Learning for Unsupervised Representation Learning on 3D Point Clouds [54.31515001741987]
  We propose a point discriminative learning method for unsupervised representation learning on 3D point clouds. We achieve this by imposing a novel point discrimination loss on the middle level and global level point features. Our method learns powerful representations and achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-08-04T15:11:48Z)
• Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
  We train a semantic point cloud segmentation network with only a small portion of points being labeled. We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples. Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
  arXiv  Detail & Related papers  (2021-07-23T14:34:57Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.