A Dataset and Benchmark for Shape Completion of Fruits for Agricultural Robotics

• URL: http://arxiv.org/abs/2407.13304v2
• Date: Tue, 17 Sep 2024 08:16:57 GMT
• Title: A Dataset and Benchmark for Shape Completion of Fruits for Agricultural Robotics
• Authors: Federico Magistri, Thomas Läbe, Elias Marks, Sumanth Nagulavancha, Yue Pan, Claus Smitt, Lasse Klingbeil, Michael Halstead, Heiner Kuhlmann, Chris McCool, Jens Behley, Cyrill Stachniss,
• Abstract summary: We propose the first publicly available 3D shape completion dataset for agricultural vision systems. We provide an RGB-D dataset for estimating the 3D shape of fruits.
• Score: 30.46518628656399
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the world population is expected to reach 10 billion by 2050, our agricultural production system needs to double its productivity despite a decline of human workforce in the agricultural sector. Autonomous robotic systems are one promising pathway to increase productivity by taking over labor-intensive manual tasks like fruit picking. To be effective, such systems need to monitor and interact with plants and fruits precisely, which is challenging due to the cluttered nature of agricultural environments causing, for example, strong occlusions. Thus, being able to estimate the complete 3D shapes of objects in presence of occlusions is crucial for automating operations such as fruit harvesting. In this paper, we propose the first publicly available 3D shape completion dataset for agricultural vision systems. We provide an RGB-D dataset for estimating the 3D shape of fruits. Specifically, our dataset contains RGB-D frames of single sweet peppers in lab conditions but also in a commercial greenhouse. For each fruit, we additionally collected high-precision point clouds that we use as ground truth. For acquiring the ground truth shape, we developed a measuring process that allows us to record data of real sweet pepper plants, both in the lab and in the greenhouse with high precision, and determine the shape of the sensed fruits. We release our dataset, consisting of almost 7,000 RGB-D frames belonging to more than 100 different fruits. We provide segmented RGB-D frames, with camera intrinsics to easily obtain colored point clouds, together with the corresponding high-precision, occlusion-free point clouds obtained with a high-precision laser scanner. We additionally enable evaluation of shape completion approaches on a hidden test set through a public challenge on a benchmark server.

Related papers

• Few-Shot Fruit Segmentation via Transfer Learning [4.616529139444651]
  We develop a few-shot semantic segmentation framework for infield fruits using transfer learning. Motivated by similar success in urban scene parsing, we propose specialized pre-training. We show that models with pre-training learn to distinguish between fruit still on the trees and fruit that have fallen on the ground.
  arXiv  Detail & Related papers  (2024-05-04T04:05:59Z)
• Generating Diverse Agricultural Data for Vision-Based Farming Applications [74.79409721178489]
  This model is capable of simulating distinct growth stages of plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture.
  arXiv  Detail & Related papers  (2024-03-27T08:42:47Z)
• Fusion-Driven Tree Reconstruction and Fruit Localization: Advancing Precision in Agriculture [2.338903291171288]
  This study introduces an innovative methodology that harnesses the synergy of RGB imagery, LiDAR, and IMU data, to achieve intricate tree reconstructions. Experiments have been carried out in both a controlled environment and an actual peach orchard.
  arXiv  Detail & Related papers  (2023-10-23T17:44:59Z)
• HarvestNet: A Dataset for Detecting Smallholder Farming Activity Using Harvest Piles and Remote Sensing [50.4506590177605]
  HarvestNet is a dataset for mapping the presence of farms in the Ethiopian regions of Tigray and Amhara during 2020-2023. We introduce a new approach based on the detection of harvest piles characteristic of many smallholder systems. We conclude that remote sensing of harvest piles can contribute to more timely and accurate cropland assessments in food insecure regions.
  arXiv  Detail & Related papers  (2023-08-23T11:03:28Z)
• Visual based Tomato Size Measurement System for an Indoor Farming Environment [3.176607626141415]
  This paper presents a size measurement method combining a machine learning model and depth images captured from three low cost RGBD cameras. The performance of the presented system is evaluated on a lab environment with real tomato fruits and fake leaves. Our three-camera system was able to achieve a height measurement accuracy of 0.9114 and a width accuracy of 0.9443.
  arXiv  Detail & Related papers  (2023-04-12T22:27:05Z)
• Agave crop segmentation and maturity classification with deep learning data-centric strategies using very high-resolution satellite imagery [101.18253437732933]
  We present an Agave tequilana Weber azul crop segmentation and maturity classification using very high resolution satellite imagery. We solve real-world deep learning problems in the very specific context of agave crop segmentation. With the resulting accurate models, agave production forecasting can be made available for large regions.
  arXiv  Detail & Related papers  (2023-03-21T03:15:29Z)
• Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots [33.21287030243106]
  Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. We address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot.
  arXiv  Detail & Related papers  (2023-03-15T20:41:24Z)
• End-to-end deep learning for directly estimating grape yield from ground-based imagery [53.086864957064876]
  This study demonstrates the application of proximal imaging combined with deep learning for yield estimation in vineyards. Three model architectures were tested: object detection, CNN regression, and transformer models. The study showed the applicability of proximal imaging and deep learning for prediction of grapevine yield on a large scale.
  arXiv  Detail & Related papers  (2022-08-04T01:34:46Z)
• Geometry-Aware Fruit Grasping Estimation for Robotic Harvesting in Orchards [6.963582954232132]
  geometry-aware network, A3N, is proposed to perform end-to-end instance segmentation and grasping estimation. We implement a global-to-local scanning strategy, which enables robots to accurately recognise and retrieve fruits in field environments. Overall, the robotic system achieves success rate of harvesting ranging from 70% - 85% in field harvesting experiments.
  arXiv  Detail & Related papers  (2021-12-08T16:17:26Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.