3D shape sensing and deep learning-based segmentation of strawberries

• URL: http://arxiv.org/abs/2111.13663v1
• Date: Fri, 26 Nov 2021 18:43:10 GMT
• Title: 3D shape sensing and deep learning-based segmentation of strawberries
• Authors: Justin Le Lou\"edec and Grzegorz Cielniak
• Abstract summary: We evaluate modern sensing technologies including stereo and time-of-flight cameras for 3D perception of shape in agriculture. We propose a novel 3D deep neural network which exploits the organised nature of information originating from the camera-based 3D sensors.
• Score: 5.634825161148484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automation and robotisation of the agricultural sector are seen as a viable solution to socio-economic challenges faced by this industry. This technology often relies on intelligent perception systems providing information about crops, plants and the entire environment. The challenges faced by traditional 2D vision systems can be addressed by modern 3D vision systems which enable straightforward localisation of objects, size and shape estimation, or handling of occlusions. So far, the use of 3D sensing was mainly limited to indoor or structured environments. In this paper, we evaluate modern sensing technologies including stereo and time-of-flight cameras for 3D perception of shape in agriculture and study their usability for segmenting out soft fruit from background based on their shape. To that end, we propose a novel 3D deep neural network which exploits the organised nature of information originating from the camera-based 3D sensors. We demonstrate the superior performance and efficiency of the proposed architecture compared to the state-of-the-art 3D networks. Through a simulated study, we also show the potential of the 3D sensing paradigm for object segmentation in agriculture and provide insights and analysis of what shape quality is needed and expected for further analysis of crops. The results of this work should encourage researchers and companies to develop more accurate and robust 3D sensing technologies to assure their wider adoption in practical agricultural applications.

Related papers

• 3D Representation Methods: A Survey [0.0]
  3D representation has experienced significant advancements, driven by the increasing demand for high-fidelity 3D models in various applications. This review examines the development and current state of 3D representation methods, highlighting their research trajectories, innovations, strength and weakness.
  arXiv  Detail & Related papers  (2024-10-09T02:01:05Z)
• Diffusion Models in 3D Vision: A Survey [11.116658321394755]
  We review the state-of-the-art approaches that leverage diffusion models for 3D visual tasks. These approaches include 3D object generation, shape completion, point cloud reconstruction, and scene understanding. We discuss potential solutions, including improving computational efficiency, enhancing multimodal fusion, and exploring the use of large-scale pretraining.
  arXiv  Detail & Related papers  (2024-10-07T04:12:23Z)
• 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities [57.444435654131006]
  3D Gaussian Splatting (3DGS) has emerged as a prominent technique with the potential to become a mainstream method for 3D representations. This survey aims to analyze existing 3DGS-related works from multiple intersecting perspectives.
  arXiv  Detail & Related papers  (2024-07-24T16:53:17Z)
• A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspective [20.798308029074786]
  3D occupancy perception technology aims to observe and understand dense 3D environments for autonomous vehicles. Similar to traditional bird's-eye view (BEV) perception, 3D occupancy perception has the nature of multi-source input and the necessity for information fusion.
  arXiv  Detail & Related papers  (2024-05-08T16:10:46Z)
• Multi-Modal Dataset Acquisition for Photometrically Challenging Object [56.30027922063559]
  This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects. We propose a novel annotation and acquisition pipeline that enhances existing 3D perception and 6D object pose datasets.
  arXiv  Detail & Related papers  (2023-08-21T10:38:32Z)
• 3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review [5.85206759397617]
  Three-dimensional (3D) point cloud analysis has become one of the attractive subjects in realistic imaging and machine visions. A significant effort has recently been devoted to developing novel strategies, using different techniques such as deep learning models. Various tasks performed on 3D point could data are investigated, including objects and scenes detection, recognition, segmentation and reconstruction.
  arXiv  Detail & Related papers  (2023-06-09T15:45:23Z)
• NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localization [80.3424839706698]
  We present NeurOCS, a framework that uses instance masks 3D boxes as input to learn 3D object shapes by means of differentiable rendering. Our approach rests on insights in learning a category-level shape prior directly from real driving scenes. We make critical design choices to learn object coordinates more effectively from an object-centric view.
  arXiv  Detail & Related papers  (2023-05-28T16:18:41Z)
• Szloca: towards a framework for full 3D tracking through a single camera in context of interactive arts [1.0878040851638]
  This research presents a novel way and a framework towards obtaining data and virtual representation of objects/people. The model does not rely on complex training of computer vision systems but combines prior computer vision research and adds a capacity to represent z depth.
  arXiv  Detail & Related papers  (2022-06-26T20:09:47Z)
• 3D Object Detection from Images for Autonomous Driving: A Survey [68.33502122185813]
  3D object detection from images is one of the fundamental and challenging problems in autonomous driving. More than 200 works have studied this problem from 2015 to 2021, encompassing a broad spectrum of theories, algorithms, and applications. We provide the first comprehensive survey of this novel and continuously growing research field, summarizing the most commonly used pipelines for image-based 3D detection.
  arXiv  Detail & Related papers  (2022-02-07T07:12:24Z)
• Active 3D Shape Reconstruction from Vision and Touch [66.08432412497443]
  Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. In 3D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings. We introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile priors to guide the shape exploration; and 3) a set of data-driven solutions with either tactile or visuo
  arXiv  Detail & Related papers  (2021-07-20T15:56:52Z)
• Kinematic 3D Object Detection in Monocular Video [123.7119180923524]
  We propose a novel method for monocular video-based 3D object detection which carefully leverages kinematic motion to improve precision of 3D localization. We achieve state-of-the-art performance on monocular 3D object detection and the Bird's Eye View tasks within the KITTI self-driving dataset.
  arXiv  Detail & Related papers  (2020-07-19T01:15:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.