Biomass phenotyping of oilseed rape through UAV multi-view oblique imaging with 3DGS and SAM model

• URL: http://arxiv.org/abs/2411.08453v1
• Date: Wed, 13 Nov 2024 09:16:21 GMT
• Title: Biomass phenotyping of oilseed rape through UAV multi-view oblique imaging with 3DGS and SAM model
• Authors: Yutao Shen, Hongyu Zhou, Xin Yang, Xuqi Lu, Ziyue Guo, Lixi Jiang, Yong He, Haiyan Cen,
• Abstract summary: This study integrates 3D Gaussian Splatting (3DGS) with the Segment Anything Model (SAM) for precise 3D reconstruction and biomass estimation of oilseed rape. 3DGS provided high accuracy, with peak signal-to-noise ratios (PSNR) of 27.43 and 29.53 and training times of 7 and 49 minutes, respectively. The SAM module achieved high segmentation accuracy, with a mean intersection over union (mIoU) of 0.961 and an F1-score of 0.980.
• Score: 18.13908148656987
• License:
• Abstract: Biomass estimation of oilseed rape is crucial for optimizing crop productivity and breeding strategies. While UAV-based imaging has advanced high-throughput phenotyping, current methods often rely on orthophoto images, which struggle with overlapping leaves and incomplete structural information in complex field environments. This study integrates 3D Gaussian Splatting (3DGS) with the Segment Anything Model (SAM) for precise 3D reconstruction and biomass estimation of oilseed rape. UAV multi-view oblique images from 36 angles were used to perform 3D reconstruction, with the SAM module enhancing point cloud segmentation. The segmented point clouds were then converted into point cloud volumes, which were fitted to ground-measured biomass using linear regression. The results showed that 3DGS (7k and 30k iterations) provided high accuracy, with peak signal-to-noise ratios (PSNR) of 27.43 and 29.53 and training times of 7 and 49 minutes, respectively. This performance exceeded that of structure from motion (SfM) and mipmap Neural Radiance Fields (Mip-NeRF), demonstrating superior efficiency. The SAM module achieved high segmentation accuracy, with a mean intersection over union (mIoU) of 0.961 and an F1-score of 0.980. Additionally, a comparison of biomass extraction models found the point cloud volume model to be the most accurate, with an determination coefficient (R2) of 0.976, root mean square error (RMSE) of 2.92 g/plant, and mean absolute percentage error (MAPE) of 6.81%, outperforming both the plot crop volume and individual crop volume models. This study highlights the potential of combining 3DGS with multi-view UAV imaging for improved biomass phenotyping.

Related papers

• Multi-modal classification of forest biodiversity potential from 2D orthophotos and 3D airborne laser scanning point clouds [47.679877727066206]
  This study investigates whether deep learning-based fusion of close-range sensing data from 2D orthophotos and 3D airborne laser scanning (ALS) point clouds can enhance biodiversity assessment. We introduce the BioVista dataset, comprising 44.378 paired samples of orthophotos and ALS point clouds from temperate forests in Denmark. Using deep neural networks (ResNet for orthophotos and PointResNet for ALS point clouds), we investigate each data modality's ability to assess forest biodiversity potential, achieving mean accuracies of 69.4% and 72.8%, respectively.
  arXiv  Detail & Related papers  (2025-01-03T09:42:25Z)
• NeFF-BioNet: Crop Biomass Prediction from Point Cloud to Drone Imagery [11.976195465657236]
  We present a biomass prediction network (BioNet) for adaptation across different data modalities, including point clouds and drone imagery. Our BioNet, utilizing a sparse 3D convolutional neural network (CNN) and a transformer-based prediction module, processes point clouds and other 3D data representations to predict biomass. To further extend BioNet for drone imagery, we integrate a neural feature field (NeFF) module, enabling 3D structure reconstruction and the transformation of 2D semantic features into the corresponding 3D surfaces.
  arXiv  Detail & Related papers  (2024-10-30T04:53:11Z)
• Dual-band feature selection for maturity classification of specialty crops by hyperspectral imaging [5.905721043072562]
  The maturity classification of specialty crops such as strawberries and tomatoes is an essential agricultural downstream activity. Recent advancements in Deep Learning have produced encouraging results in color images for maturity classification applications. We present a feature extraction method to compute yet distinctive features for the maturity classification.
  arXiv  Detail & Related papers  (2024-05-16T10:01:16Z)
• Mapping Walnut Water Stress with High Resolution Multispectral UAV Imagery and Machine Learning [0.0]
  This study presents a machine learning approach using Random Forest (RF) models to map stem water potential (SWP) From 2017 to 2018, five flights of an UAV equipped with a seven-band multispectral camera were conducted over a commercial walnut orchard. RF regression model, utilizing vegetation indices derived from orthomosaiced UAV imagery and weather data, effectively estimated ground-measured SWPs. RF classification model predicted water stress levels in walnut trees with 85% accuracy, surpassing the 80% accuracy of the reduced classification model.
  arXiv  Detail & Related papers  (2023-12-30T02:58:45Z)
• Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network [63.845552349914186]
  Capturing global contextual information plays a critical role in breast ultrasound (BUS) image classification. Vision Transformers have an improved capability of capturing global contextual information but may distort the local image patterns due to the tokenization operations. In this study, we proposed a hybrid multitask deep neural network called Hybrid-MT-ESTAN, designed to perform BUS tumor classification and segmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:19:32Z)
• ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis [99.06490355990354]
  We propose ConsistentNeRF, a method that leverages depth information to regularize both multi-view and single-view 3D consistency among pixels. Our approach can considerably enhance model performance in sparse view conditions, achieving improvements of up to 94% in PSNR, in SSIM, and 31% in LPIPS.
  arXiv  Detail & Related papers  (2023-05-18T15:18:01Z)
• Generating high-quality 3DMPCs by adaptive data acquisition and NeREF-based radiometric calibration with UGV plant phenotyping system [3.7387019397567793]
  This study proposed a novel approach for adaptive data acquisition and radiometric calibration to generate high-quality 3DMPCs of plants. The integrity of the whole-plant data was improved by an average of 23.6% compared to the fixed viewpoints alone. The 3D-calibrated plant 3DMPCs improved the predictive accuracy of PLSR for chlorophyll content, with an average increase of 0.07 in R2 and an average decrease of 21.25% in RMSE.
  arXiv  Detail & Related papers  (2023-05-11T12:59:21Z)
• Federated Learning Enables Big Data for Rare Cancer Boundary Detection [98.5549882883963]
  We present findings from the largest Federated ML study to-date, involving data from 71 healthcare institutions across 6 continents. We generate an automatic tumor boundary detector for the rare disease of glioblastoma. We demonstrate a 33% improvement over a publicly trained model to delineate the surgically targetable tumor, and 23% improvement over the tumor's entire extent.
  arXiv  Detail & Related papers  (2022-04-22T17:27:00Z)
• A Multi-Stage model based on YOLOv3 for defect detection in PV panels based on IR and Visible Imaging by Unmanned Aerial Vehicle [65.99880594435643]
  We propose a novel model to detect panel defects on aerial images captured by unmanned aerial vehicle. The model combines detections of panels and defects to refine its accuracy. The proposed model has been validated on two big PV plants in the south of Italy.
  arXiv  Detail & Related papers  (2021-11-23T08:04:32Z)
• Deep learning of multi-resolution X-Ray micro-CT images for multi-scale modelling [0.0]
  We develop a 3D Enhanced Deep Super Resolution (EDSR) convolutional neural network to create enhanced, high-resolution data over large spatial scales. We validate the network with various metrics: textual analysis, segmentation behaviour and pore-network model (PNM) multiphase flow simulations. The EDSR generated model is more accurate than the base LR model at predicting experimental behaviour in the presence of heterogeneities.
  arXiv  Detail & Related papers  (2021-11-01T21:49:22Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.