Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots

• URL: http://arxiv.org/abs/1912.12555v1
• Date: Sun, 29 Dec 2019 00:30:59 GMT
• Title: Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots
• Authors: Hanwen Kang and Chao Chen
• Abstract summary: This paper develops a framework of visual perception and modelling for robotic harvesting of fruits in the orchard environments. The framework includes visual perception, scenarios mapping, and fruit modelling. Experiment results show that visual perception and modelling algorithm can accurately detect and localise the fruits.
• Score: 6.634537400804884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision perception and modelling are the essential tasks of robotic harvesting in the unstructured orchard. This paper develops a framework of visual perception and modelling for robotic harvesting of fruits in the orchard environments. The developed framework includes visual perception, scenarios mapping, and fruit modelling. The Visual perception module utilises a deep-learning model to perform multi-purpose visual perception task within the working scenarios; The scenarios mapping module applies OctoMap to represent the multiple classes of objects or elements within the environment; The fruit modelling module estimates the geometry property of objects and estimates the proper access pose of each fruit. The developed framework is implemented and evaluated in the apple orchards. The experiment results show that visual perception and modelling algorithm can accurately detect and localise the fruits, and modelling working scenarios in real orchard environments. The $F_{1}$ score and mean intersection of union of visual perception module on fruit detection and segmentation are 0.833 and 0.852, respectively. The accuracy of the fruit modelling in terms of centre localisation and pose estimation are 0.955 and 0.923, respectively. Overall, an accurate visual perception and modelling algorithm are presented in this paper.

Related papers

• Evaluating Multiview Object Consistency in Humans and Image Models [68.36073530804296]
  We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape. We collect 35K trials of behavioral data from over 500 participants. We then evaluate the performance of common vision models.
  arXiv  Detail & Related papers  (2024-09-09T17:59:13Z)
• Zero-Shot Object-Centric Representation Learning [72.43369950684057]
  We study current object-centric methods through the lens of zero-shot generalization. We introduce a benchmark comprising eight different synthetic and real-world datasets. We find that training on diverse real-world images improves transferability to unseen scenarios.
  arXiv  Detail & Related papers  (2024-08-17T10:37:07Z)
• FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework [5.363729942767801]
  We introduce FruitNeRF, a unified novel fruit counting framework. We use state-of-the-art view synthesis methods to count any fruit type directly in 3D. We evaluate our methodology using both real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2024-08-12T14:40:38Z)
• A Control-Centric Benchmark for Video Prediction [69.22614362800692]
  We propose a benchmark for action-conditioned video prediction in the form of a control benchmark. Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions. We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
  arXiv  Detail & Related papers  (2023-04-26T17:59:45Z)
• Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots [33.21287030243106]
  Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. We address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot.
  arXiv  Detail & Related papers  (2023-03-15T20:41:24Z)
• MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [84.80956484848505]
  MegaPose is a method to estimate the 6D pose of novel objects, that is, objects unseen during training. We present a 6D pose refiner based on a render&compare strategy which can be applied to novel objects. Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner.
  arXiv  Detail & Related papers  (2022-12-13T19:30:03Z)
• NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction [11.45602594277673]
  We present a novel viewpoint planning approach that explicitly uses information about the predicted fruit shapes to compute targeted viewpoints. We also show the viability of our approach for mapping sweet peppers with a real robotic system in a commercial glasshouse.
  arXiv  Detail & Related papers  (2022-09-30T11:09:54Z)
• End-to-end deep learning for directly estimating grape yield from ground-based imagery [53.086864957064876]
  This study demonstrates the application of proximal imaging combined with deep learning for yield estimation in vineyards. Three model architectures were tested: object detection, CNN regression, and transformer models. The study showed the applicability of proximal imaging and deep learning for prediction of grapevine yield on a large scale.
  arXiv  Detail & Related papers  (2022-08-04T01:34:46Z)
• A methodology for detection and localization of fruits in apples orchards from aerial images [0.0]
  This work presents a methodology for automated fruit counting employing aerial-images. It includes algorithms based on multiple view geometry to perform fruits tracking. Preliminary assessments show correlations above 0.8 between fruit counting and true yield for apples.
  arXiv  Detail & Related papers  (2021-10-24T01:57:52Z)
• Visual Distant Supervision for Scene Graph Generation [66.10579690929623]
  Scene graph models usually require supervised learning on large quantities of labeled data with intensive human annotation. We propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data. Comprehensive experimental results show that our distantly supervised model outperforms strong weakly supervised and semi-supervised baselines.
  arXiv  Detail & Related papers  (2021-03-29T06:35:24Z)
• Real-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting [6.634537400804884]
  The framework includes a multi-function neural network for fruit recognition and a Pointnet grasp estimation. The proposed framework can accurately localise and estimate the grasp pose for robotic grasping.
  arXiv  Detail & Related papers  (2020-03-30T09:37:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.