FruitNeRF++: A Generalized Multi-Fruit Counting Method Utilizing Contrastive Learning and Neural Radiance Fields

• URL: http://arxiv.org/abs/2505.19863v1
• Date: Mon, 26 May 2025 11:48:22 GMT
• Title: FruitNeRF++: A Generalized Multi-Fruit Counting Method Utilizing Contrastive Learning and Neural Radiance Fields
• Authors: Lukas Meyer, Andrei-Timotei Ardelean, Tim Weyrich, Marc Stamminger,
• Abstract summary: We introduce FruitNeRF++, a novel fruit-counting approach that combines contrastive learning with neural radiance fields.<n>Our work is based on FruitNeRF, which employs a neural semantic field combined with a fruit-specific clustering approach.<n>Our results demonstrate that FruitNeRF++ is easier to control and compares favorably to other state-of-the-art methods.
• Score: 5.669562773066833
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce FruitNeRF++, a novel fruit-counting approach that combines contrastive learning with neural radiance fields to count fruits from unstructured input photographs of orchards. Our work is based on FruitNeRF, which employs a neural semantic field combined with a fruit-specific clustering approach. The requirement for adaptation for each fruit type limits the applicability of the method, and makes it difficult to use in practice. To lift this limitation, we design a shape-agnostic multi-fruit counting framework, that complements the RGB and semantic data with instance masks predicted by a vision foundation model. The masks are used to encode the identity of each fruit as instance embeddings into a neural instance field. By volumetrically sampling the neural fields, we extract a point cloud embedded with the instance features, which can be clustered in a fruit-agnostic manner to obtain the fruit count. We evaluate our approach using a synthetic dataset containing apples, plums, lemons, pears, peaches, and mangoes, as well as a real-world benchmark apple dataset. Our results demonstrate that FruitNeRF++ is easier to control and compares favorably to other state-of-the-art methods.

Related papers

• Horticultural Temporal Fruit Monitoring via 3D Instance Segmentation and Re-Identification using Colored Point Clouds [29.23207854514898]
  We propose a novel method for fruit instance segmentation and re-identification on 3D terrestrial point clouds collected over time.<n>Our approach directly operates on dense colored point clouds, capturing fine-grained 3D spatial detail.<n>We evaluate our approach on real-world datasets of strawberries and apples, demonstrating that it outperforms existing methods in both instance segmentation and temporal re-identification.
  arXiv  Detail & Related papers  (2024-11-12T13:53:22Z)
• FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework [5.363729942767801]
  We introduce FruitNeRF, a unified novel fruit counting framework. We use state-of-the-art view synthesis methods to count any fruit type directly in 3D. We evaluate our methodology using both real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2024-08-12T14:40:38Z)
• A pipeline for multiple orange detection and tracking with 3-D fruit relocalization and neural-net based yield regression in commercial citrus orchards [0.0]
  We propose a non-invasive alternative that utilizes fruit counting from videos, implemented as a pipeline. To handle occluded and re-appeared fruit, we introduce a relocalization component that employs 3-D estimation of fruit locations. By ensuring that at least 30% of the fruit is accurately detected, tracked, and counted, our yield regressor achieves an impressive coefficient of determination of 0.85.
  arXiv  Detail & Related papers  (2023-12-27T21:22:43Z)
• Fruit Ripeness Classification: a Survey [59.11160990637616]
  Many automatic methods have been proposed that employ a variety of feature descriptors for the food item to be graded. Machine learning and deep learning techniques dominate the top-performing methods. Deep learning can operate on raw data and thus relieve the users from having to compute complex engineered features.
  arXiv  Detail & Related papers  (2022-12-29T19:32:20Z)
• Extracting Semantic Knowledge from GANs with Unsupervised Learning [65.32631025780631]
  Generative Adversarial Networks (GANs) encode semantics in feature maps in a linearly separable form. We propose a novel clustering algorithm, named KLiSH, which leverages the linear separability to cluster GAN's features. KLiSH succeeds in extracting fine-grained semantics of GANs trained on datasets of various objects.
  arXiv  Detail & Related papers  (2022-11-30T03:18:16Z)
• Apple Counting using Convolutional Neural Networks [22.504279159923765]
  Estimating accurate and reliable fruit and vegetable counts from images in real-world settings, such as orchards, is a challenging problem. We formulate fruit counting from images as a multi-class classification problem and solve it by training a Convolutional Neural Network. Our network outperforms it in three out of four datasets with a maximum of 94% accuracy.
  arXiv  Detail & Related papers  (2022-08-24T14:13:40Z)
• Facilitated machine learning for image-based fruit quality assessment in developing countries [68.8204255655161]
  Automated image classification is a common task for supervised machine learning in food science. We propose an alternative method based on pre-trained vision transformers (ViTs) It can be easily implemented with limited resources on a standard device.
  arXiv  Detail & Related papers  (2022-07-10T19:52:20Z)
• A methodology for detection and localization of fruits in apples orchards from aerial images [0.0]
  This work presents a methodology for automated fruit counting employing aerial-images. It includes algorithms based on multiple view geometry to perform fruits tracking. Preliminary assessments show correlations above 0.8 between fruit counting and true yield for apples.
  arXiv  Detail & Related papers  (2021-10-24T01:57:52Z)
• Convolutional Neural Networks from Image Markers [62.997667081978825]
  Feature Learning from Image Markers (FLIM) was recently proposed to estimate convolutional filters, with no backpropagation, from strokes drawn by a user on very few images. This paper extends FLIM for fully connected layers and demonstrates it on different image classification problems. The results show that FLIM-based convolutional neural networks can outperform the same architecture trained from scratch by backpropagation.
  arXiv  Detail & Related papers  (2020-12-15T22:58:23Z)
• Permutation Matters: Anisotropic Convolutional Layer for Learning on Point Clouds [145.79324955896845]
  We propose a permutable anisotropic convolutional operation (PAI-Conv) that calculates soft-permutation matrices for each point. Experiments on point clouds demonstrate that PAI-Conv produces competitive results in classification and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2020-05-27T02:42:29Z)
• Linguistically Driven Graph Capsule Network for Visual Question Reasoning [153.76012414126643]
  We propose a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network" The compositional process is guided by the linguistic parse tree. Specifically, we bind each capsule in the lowest layer to bridge the linguistic embedding of a single word in the original question with visual evidence. Experiments on the CLEVR dataset, CLEVR compositional generation test, and FigureQA dataset demonstrate the effectiveness and composition generalization ability of our end-to-end model.
  arXiv  Detail & Related papers  (2020-03-23T03:34:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.