Semantics-Aware Next-best-view Planning for Efficient Search and Detection of Task-relevant Plant Parts

• URL: http://arxiv.org/abs/2306.09801v2
• Date: Thu, 9 May 2024 20:52:21 GMT
• Title: Semantics-Aware Next-best-view Planning for Efficient Search and Detection of Task-relevant Plant Parts
• Authors: Akshay K. Burusa, Joost Scholten, David Rapado Rincon, Xin Wang, Eldert J. van Henten, Gert Kootstra,
• Abstract summary: To automate harvesting and de-leafing of tomato plants, it is important to search and detect the task-relevant plant parts. Current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts. We propose a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts.
• Score: 3.9074818653555554
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To automate harvesting and de-leafing of tomato plants using robots, it is important to search and detect the task-relevant plant parts. This is challenging due to high levels of occlusion in tomato plants. Active vision is a promising approach to viewpoint planning, which helps robots to deliberately plan camera viewpoints to overcome occlusion and improve perception accuracy. However, current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts and spend time on perceiving irrelevant plant parts, making them inefficient for targeted perception. We propose a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts and prioritise them during view planning. We evaluated our strategy on the task of searching and detecting the relevant plant parts using simulation and real-world experiments. In simulation, using 3D models of tomato plants with varying structural complexity, our semantics-aware strategy could search and detect 81.8% of all the relevant plant parts using nine viewpoints. It was significantly faster and detected more plant parts than predefined, random, and volumetric active-vision strategies. Our strategy was also robust to uncertainty in plant and plant-part position, plant complexity, and different viewpoint-sampling strategies. Further, in real-world experiments, our strategy could search and detect 82.7% of all the relevant plant parts using seven viewpoints, under real-world conditions with natural variation and occlusion, natural illumination, sensor noise, and uncertainty in camera poses. Our results clearly indicate the advantage of using semantics-aware active vision for targeted perception of plant parts and its applicability in real-world setups. We believe that it can significantly improve the speed and robustness of automated harvesting and de-leafing in tomato crop production.

Related papers

• AI planning in the imagination: High-level planning on learned abstract search spaces [68.75684174531962]
  We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training. We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
  arXiv  Detail & Related papers  (2023-08-16T22:47:16Z)
• Improving Data Efficiency for Plant Cover Prediction with Label Interpolation and Monte-Carlo Cropping [7.993547048820065]
  The plant community composition is an essential indicator of environmental changes and is usually analyzed in ecological field studies. We introduce an approach to interpolate the sparse labels in the collected vegetation plot time series down to the intermediate dense and unlabeled images. We also introduce a new method we call Monte-Carlo Cropping to deal with high-resolution images efficiently.
  arXiv  Detail & Related papers  (2023-07-17T15:17:39Z)
• Push Past Green: Learning to Look Behind Plant Foliage by Moving It [19.36396157137122]
  Partial visibility, extreme clutter, thin structures, and unknown geometry and dynamics for plants make such manipulation challenging. We use self-supervision to train SRPNet, a neural network that predicts what space is revealed on execution of a candidate action on a given plant. As SRPNet does not just predict how much space is revealed but also where it is revealed, we can execute a sequence of actions that incrementally reveal more and more space beneath the plant foliage.
  arXiv  Detail & Related papers  (2023-07-06T17:55:28Z)
• PlantDet: A benchmark for Plant Detection in the Three-Rivers-Source Region [4.676030127116814]
  We construct a dataset for Plant detection in the Three-River-Source region (PTRS) It comprises 21 types, 6965 high-resolution images of 2160*3840 pixels, captured by diverse sensors and platforms, and featuring objects of varying shapes and sizes. The PTRS presents us with challenges such as dense occlusion, varying leaf resolutions, and high feature similarity among plants, prompting us to develop a novel object detection network named PlantDet.
  arXiv  Detail & Related papers  (2023-04-11T04:18:56Z)
• Development and evaluation of automated localisation and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking [0.0]
  This paper presents a novel approach for building generic representations in occluded agro-food environments. It is based on a detection algorithm that generates partial point clouds for each detected object, followed by a 3D multi-object tracking algorithm. The accuracy of the representation was evaluated in a real-world environment, where successful representation and localisation of tomatoes in tomato plants were achieved.
  arXiv  Detail & Related papers  (2022-11-04T21:51:53Z)
• Semantic Image Segmentation with Deep Learning for Vine Leaf Phenotyping [59.0626764544669]
  In this study, we use Deep Learning methods to semantically segment grapevine leaves images in order to develop an automated object detection system for leaf phenotyping. Our work contributes to plant lifecycle monitoring through which dynamic traits such as growth and development can be captured and quantified.
  arXiv  Detail & Related papers  (2022-10-24T14:37:09Z)
• Incremental 3D Scene Completion for Safe and Efficient Exploration Mapping and Planning [60.599223456298915]
  We propose a novel way to integrate deep learning into exploration by leveraging 3D scene completion for informed, safe, and interpretable mapping and planning. We show that our method can speed up coverage of an environment by 73% compared to the baselines with only minimal reduction in map accuracy. Even if scene completions are not included in the final map, we show that they can be used to guide the robot to choose more informative paths, speeding up the measurement of the scene with the robot's sensors by 35%.
  arXiv  Detail & Related papers  (2022-08-17T14:19:33Z)
• Attention-driven Next-best-view Planning for Efficient Reconstruction of Plants and Targeted Plant Parts [0.0]
  We investigate the role of attention in improving targeted perception using an attention-driven NBV planning strategy. We show that focusing attention on task-relevant plant parts can significantly improve the speed and accuracy of 3D reconstruction.
  arXiv  Detail & Related papers  (2022-06-21T11:46:57Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis [78.26022688167133]
  We present a large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis. The proposed dataset contains 100,000 images and 25 different object types. We also propose a new layout-weighted performance metric alongside the dataset for evaluating object detection and segmentation performance.
  arXiv  Detail & Related papers  (2021-12-29T17:23:24Z)
• Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection [60.83360138070649]
  The paper presents an approach for analyzing aerial images of a potato crop using deep neural networks. The main objective is to demonstrate automated spatial recognition of a healthy versus stressed crop at a plant level. Experimental validation demonstrated the ability for distinguishing healthy and stressed plants in field images, achieving an average Dice coefficient of 0.74.
  arXiv  Detail & Related papers  (2021-06-14T21:57:40Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.