NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction

• URL: http://arxiv.org/abs/2209.15376v3
• Date: Wed, 30 Aug 2023 11:04:14 GMT
• Title: NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction
• Authors: Rohit Menon and Tobias Zaenker and Nils Dengler and Maren Bennewitz
• Abstract summary: We present a novel viewpoint planning approach that explicitly uses information about the predicted fruit shapes to compute targeted viewpoints. We also show the viability of our approach for mapping sweet peppers with a real robotic system in a commercial glasshouse.
• Score: 11.45602594277673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Active perception for fruit mapping and harvesting is a difficult task since occlusions occur frequently and the location as well as size of fruits change over time. State-of-the-art viewpoint planning approaches utilize computationally expensive ray casting operations to find good viewpoints aiming at maximizing information gain and covering the fruits in the scene. In this paper, we present a novel viewpoint planning approach that explicitly uses information about the predicted fruit shapes to compute targeted viewpoints that observe as yet unobserved parts of the fruits. Furthermore, we formulate the concept of viewpoint dissimilarity to reduce the sampling space for more efficient selection of useful, dissimilar viewpoints. Our simulation experiments with a UR5e arm equipped with an RGB-D sensor provide a quantitative demonstration of the efficacy of our iterative next best view planning method based on shape completion. In comparative experiments with a state-of-the-art viewpoint planner, we demonstrate improvement not only in the estimation of the fruit sizes, but also in their reconstruction, while significantly reducing the planning time. Finally, we show the viability of our approach for mapping sweet peppers plants with a real robotic system in a commercial glasshouse.

Related papers

• Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image Retrieval [85.73149096516543]
  We address the choice of viewpoint during sketch creation in Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) A pilot study highlights the system's struggle when query-sketches differ in viewpoint from target instances. To reconcile this, we advocate for a view-aware system, seamlessly accommodating both view-agnostic and view-specific tasks.
  arXiv  Detail & Related papers  (2024-07-01T21:20:44Z)
• A pipeline for multiple orange detection and tracking with 3-D fruit relocalization and neural-net based yield regression in commercial citrus orchards [0.0]
  We propose a non-invasive alternative that utilizes fruit counting from videos, implemented as a pipeline. To handle occluded and re-appeared fruit, we introduce a relocalization component that employs 3-D estimation of fruit locations. By ensuring that at least 30% of the fruit is accurately detected, tracked, and counted, our yield regressor achieves an impressive coefficient of determination of 0.85.
  arXiv  Detail & Related papers  (2023-12-27T21:22:43Z)
• Gradient-based Local Next-best-view Planning for Improved Perception of Targeted Plant Nodes [0.0]
  We formulate this problem as a local next-best-view (NBV) planning task. Our formulation focuses on quickly improving the perception accuracy of a single target node to maximise its chances of being cut. We propose a gradient-based NBV planner using differential ray sampling, which directly estimates the local gradient direction for viewpoint planning.
  arXiv  Detail & Related papers  (2023-11-28T13:02:33Z)
• Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots [33.21287030243106]
  Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. We address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot.
  arXiv  Detail & Related papers  (2023-03-15T20:41:24Z)
• Monocular BEV Perception of Road Scenes via Front-to-Top View Projection [57.19891435386843]
  We present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view. Our model runs at 25 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction.
  arXiv  Detail & Related papers  (2022-11-15T13:52:41Z)
• Visual Learning-based Planning for Continuous High-Dimensional POMDPs [81.16442127503517]
  Visual Tree Search (VTS) is a learning and planning procedure that combines generative models learned offline with online model-based POMDP planning. VTS bridges offline model training and online planning by utilizing a set of deep generative observation models to predict and evaluate the likelihood of image observations in a Monte Carlo tree search planner. We show that VTS is robust to different observation noises and, since it utilizes online, model-based planning, can adapt to different reward structures without the need to re-train.
  arXiv  Detail & Related papers  (2021-12-17T11:53:31Z)
• PANet: Perspective-Aware Network with Dynamic Receptive Fields and Self-Distilling Supervision for Crowd Counting [63.84828478688975]
  We propose a novel perspective-aware approach called PANet to address the perspective problem. Based on the observation that the size of the objects varies greatly in one image due to the perspective effect, we propose the dynamic receptive fields (DRF) framework. The framework is able to adjust the receptive field by the dilated convolution parameters according to the input image, which helps the model to extract more discriminative features for each local region.
  arXiv  Detail & Related papers  (2021-10-31T04:43:05Z)
• Measuring the Ripeness of Fruit with Hyperspectral Imaging and Deep Learning [14.853897011640022]
  We present a system to measure the ripeness of fruit with a hyperspectral camera and a suitable deep neural network architecture. This architecture did outperform competitive baseline models on the prediction of the state of ripeness.
  arXiv  Detail & Related papers  (2021-04-20T07:43:19Z)
• Plan2Vec: Unsupervised Representation Learning by Latent Plans [106.37274654231659]
  We introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning. Plan2vec constructs a weighted graph on an image dataset using near-neighbor distances, and then extrapolates this local metric to a global embedding by distilling path-integral over planned path. We demonstrate the effectiveness of plan2vec on one simulated and two challenging real-world image datasets.
  arXiv  Detail & Related papers  (2020-05-07T17:52:23Z)
• Hallucinative Topological Memory for Zero-Shot Visual Planning [86.20780756832502]
  In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline. Most previous works on VP approached the problem by planning in a learned latent space, resulting in low-quality visual plans. Here, we propose a simple VP method that plans directly in image space and displays competitive performance.
  arXiv  Detail & Related papers  (2020-02-27T18:54:42Z)
• Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots [6.634537400804884]
  This paper develops a framework of visual perception and modelling for robotic harvesting of fruits in the orchard environments. The framework includes visual perception, scenarios mapping, and fruit modelling. Experiment results show that visual perception and modelling algorithm can accurately detect and localise the fruits.
  arXiv  Detail & Related papers  (2019-12-29T00:30:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.