3D Reconstruction and Information Fusion between Dormant and Canopy Seasons in Commercial Orchards Using Deep Learning and Fast GICP

• URL: http://arxiv.org/abs/2507.01912v1
• Date: Wed, 02 Jul 2025 17:24:18 GMT
• Title: 3D Reconstruction and Information Fusion between Dormant and Canopy Seasons in Commercial Orchards Using Deep Learning and Fast GICP
• Authors: Ranjan Sapkota, Zhichao Meng, Martin Churuvija, Xiaoqiang Du, Zenghong Ma, Manoj Karkee,
• Abstract summary: In orchard automation, dense foliage during the canopy season severely occludes tree structures.<n> canopy structure is more open and visible during the dormant season when trees are defoliated.<n>We present an information fusion framework that integrates multi-seasonal structural data to support robotic and automated crop load management.
• Score: 0.9565934024763958
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In orchard automation, dense foliage during the canopy season severely occludes tree structures, minimizing visibility to various canopy parts such as trunks and branches, which limits the ability of a machine vision system. However, canopy structure is more open and visible during the dormant season when trees are defoliated. In this work, we present an information fusion framework that integrates multi-seasonal structural data to support robotic and automated crop load management during the entire growing season. The framework combines high-resolution RGB-D imagery from both dormant and canopy periods using YOLOv9-Seg for instance segmentation, Kinect Fusion for 3D reconstruction, and Fast Generalized Iterative Closest Point (Fast GICP) for model alignment. Segmentation outputs from YOLOv9-Seg were used to extract depth-informed masks, which enabled accurate 3D point cloud reconstruction via Kinect Fusion; these reconstructed models from each season were subsequently aligned using Fast GICP to achieve spatially coherent multi-season fusion. The YOLOv9-Seg model, trained on manually annotated images, achieved a mean squared error (MSE) of 0.0047 and segmentation mAP@50 scores up to 0.78 for trunks in dormant season dataset. Kinect Fusion enabled accurate reconstruction of tree geometry, validated with field measurements resulting in root mean square errors (RMSE) of 5.23 mm for trunk diameter, 4.50 mm for branch diameter, and 13.72 mm for branch spacing. Fast GICP achieved precise cross-seasonal registration with a minimum fitness score of 0.00197, allowing integrated, comprehensive tree structure modeling despite heavy occlusions during the growing season. This fused structural representation enables robotic systems to access otherwise obscured architectural information, improving the precision of pruning, thinning, and other automated orchard operations.

Related papers

• Three-dimentional reconstruction of complex, dynamic population canopy architecture for crops with a novel point cloud completion model: A case study in Brassica napus rapeseed [4.377318975816766]
  We propose an effective method for 3D reconstruction of complex, dynamic population canopy architecture for rapeseed crops.<n>A complete point cloud generation framework was developed for automated annotation of the training dataset.<n>The crop population point cloud completion network (CP-PCN) was then designed with a multi-resolution dynamic graph convolutional encoder (MRDG) and a point pyramid decoder (PPD) to predict occluded points.
  arXiv  Detail & Related papers  (2025-06-23T05:02:31Z)
• Adaptive Per-Tree Canopy Volume Estimation Using Mobile LiDAR in Structured and Unstructured Orchards [42.32889225423819]
  We present a real-time system for per-tree canopy volume estimation using mobile LiDAR data collected during routine robotic navigation.<n>We evaluate the system across two commercial orchards, one pistachio orchard with regular spacing and one almond orchard with dense, overlapping crowns.
  arXiv  Detail & Related papers  (2025-06-09T08:40:28Z)
• Estimating the Diameter at Breast Height of Trees in a Forest With a Single 360 Camera [52.85399274741336]
  Forest inventories rely on accurate measurements of the diameter at breast height (DBH) for ecological monitoring, resource management, and carbon accounting.<n>While LiDAR-based techniques can achieve centimeter-level precision, they are cost-prohibitive and operationally complex.<n>We present a low-cost alternative that only needs a consumer-grade 360 video camera.
  arXiv  Detail & Related papers  (2025-05-06T01:09:07Z)
• A Novel WaveInst-based Network for Tree Trunk Structure Extraction and Pattern Analysis in Forest Inventory [2.793797265684592]
  This work proposes a novel WaveInst instance segmentation framework, involving a discrete wavelet transform, to improve tree structure extraction.<n> Experimental results of the proposed model show superior performance on SynthTree43k, CaneTree100, Urban Street and our PoplarDataset.<n>The proposed method achieves a mean average precision of 49.6 and 24.3 for the structure extraction of mature and juvenile trees, respectively, surpassing the existing state-of-the-art method by 9.9.
  arXiv  Detail & Related papers  (2025-05-03T02:19:55Z)
• Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata [70.9375320609781]
  We aim to generate fine-grained 3D geometry from large-scale sparse LiDAR scans, abundantly captured by autonomous vehicles (AV) We propose hierarchical Generative Cellular Automata (hGCA), a spatially scalable 3D generative model, which grows geometry with local kernels following, in a coarse-to-fine manner, equipped with a light-weight planner to induce global consistency.
  arXiv  Detail & Related papers  (2024-06-12T14:56:56Z)
• FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models [54.787308652357794]
  FinerCut is a new form of fine-grained layer pruning for transformer networks. Our approach retains 90% performance of Llama3-8B with 25% layers removed, and 95% performance of Llama3-70B with 30% layers removed, all without fine-tuning or post-pruning reconstruction.
  arXiv  Detail & Related papers  (2024-05-28T14:21:15Z)
• Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [66.3814684757376]
  This work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level. The outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.
  arXiv  Detail & Related papers  (2024-03-21T10:38:18Z)
• CherryPicker: Semantic Skeletonization and Topological Reconstruction of Cherry Trees [3.8697834534260447]
  We present CherryPicker, an automatic pipeline that reconstructs photo-metric point clouds of trees. Our system combines several state-of-the-art algorithms to enable automatic processing for further usage in 3D-plant phenotyping applications.
  arXiv  Detail & Related papers  (2023-04-10T16:54:05Z)
• Approach for modeling single branches of meadow orchard trees with 3D point clouds [0.0]
  The cultivation of orchard meadows provides an ecological benefit for biodiversity, which is significantly higher than in intensively cultivated orchards. The goal of this research is to create a tree model to automatically determine possible pruning points for stand-alone trees within meadows.
  arXiv  Detail & Related papers  (2021-04-12T08:25:27Z)
• GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision [65.13980934546957]
  We present a novel end-to-end framework named as GSNet (Geometric and Scene-aware Network) It jointly estimates 6DoF poses and reconstructs detailed 3D car shapes from single urban street view. We evaluate GSNet on the largest multi-task ApolloCar3D benchmark and achieve state-of-the-art performance both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-07-26T13:05:55Z)
• SparseFusion: Dynamic Human Avatar Modeling from Sparse RGBD Images [49.52782544649703]
  We propose a novel approach to reconstruct 3D human body shapes based on a sparse set of RGBD frames. The main challenge is how to robustly fuse these sparse frames into a canonical 3D model. Our framework is flexible, with potential applications going beyond shape reconstruction.
  arXiv  Detail & Related papers  (2020-06-05T18:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.