Wheat3DGS: In-field 3D Reconstruction, Instance Segmentation and Phenotyping of Wheat Heads with Gaussian Splatting

• URL: http://arxiv.org/abs/2504.06978v1
• Date: Wed, 09 Apr 2025 15:31:42 GMT
• Title: Wheat3DGS: In-field 3D Reconstruction, Instance Segmentation and Phenotyping of Wheat Heads with Gaussian Splatting
• Authors: Daiwei Zhang, Joaquin Gajardo, Tomislav Medic, Isinsu Katircioglu, Mike Boss, Norbert Kirchgessner, Achim Walter, Lukas Roth,
• Abstract summary: We present Wheat3DGS, a novel approach that leverages 3DGS and the Segment Anything Model (SAM) for precise 3D instance segmentation and morphological measurement of hundreds of wheat heads automatically.<n>We validate the accuracy of wheat breeding head extraction against high-resolution laser scan data, obtaining per-instance mean absolute percentage errors of 15.1%, 18.3%, and 40.2% for length, width, and volume.
• Score: 1.4100451538155885
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated extraction of plant morphological traits is crucial for supporting crop breeding and agricultural management through high-throughput field phenotyping (HTFP). Solutions based on multi-view RGB images are attractive due to their scalability and affordability, enabling volumetric measurements that 2D approaches cannot directly capture. While advanced methods like Neural Radiance Fields (NeRFs) have shown promise, their application has been limited to counting or extracting traits from only a few plants or organs. Furthermore, accurately measuring complex structures like individual wheat heads-essential for studying crop yields-remains particularly challenging due to occlusions and the dense arrangement of crop canopies in field conditions. The recent development of 3D Gaussian Splatting (3DGS) offers a promising alternative for HTFP due to its high-quality reconstructions and explicit point-based representation. In this paper, we present Wheat3DGS, a novel approach that leverages 3DGS and the Segment Anything Model (SAM) for precise 3D instance segmentation and morphological measurement of hundreds of wheat heads automatically, representing the first application of 3DGS to HTFP. We validate the accuracy of wheat head extraction against high-resolution laser scan data, obtaining per-instance mean absolute percentage errors of 15.1%, 18.3%, and 40.2% for length, width, and volume. We provide additional comparisons to NeRF-based approaches and traditional Muti-View Stereo (MVS), demonstrating superior results. Our approach enables rapid, non-destructive measurements of key yield-related traits at scale, with significant implications for accelerating crop breeding and improving our understanding of wheat development.

Related papers

• BBoxCut: A Targeted Data Augmentation Technique for Enhancing Wheat Head Detection Under Occlusions [2.249916681499244]
  We propose a novel data augmentation technique, BBoxCut, which uses random localized masking to simulate occlusions caused by leaves and neighboring wheat heads.<n>Our augmentation technique led to significant improvements both qualitatively and quantitatively.<n>In particular, the improvements were particularly evident in scenarios involving occluded wheat heads.
  arXiv  Detail & Related papers  (2025-03-31T12:59:02Z)
• AgriField3D: A Curated 3D Point Cloud and Procedural Model Dataset of Field-Grown Maize from a Diversity Panel [12.89812013060155]
  AgriField3D is a curated dataset of 3D point clouds of field-grown maize plants from a diverse genetic panel.<n>Our dataset comprises over 1,000 high-quality point clouds collected using a Terrestrial Laser Scanner.
  arXiv  Detail & Related papers  (2025-03-10T19:53:20Z)
• DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models [67.50989119438508]
  We introduce DSplats, a novel method that directly denoises multiview images using Gaussian-based Reconstructors to produce realistic 3D assets. Our experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction.
  arXiv  Detail & Related papers  (2024-12-11T07:32:17Z)
• MonoGSDF: Exploring Monocular Geometric Cues for Gaussian Splatting-Guided Implicit Surface Reconstruction [84.07233691641193]
  We introduce MonoGSDF, a novel method that couples primitives with a neural Signed Distance Field (SDF) for high-quality reconstruction.<n>To handle arbitrary-scale scenes, we propose a scaling strategy for robust generalization.<n>Experiments on real-world datasets outperforms prior methods while maintaining efficiency.
  arXiv  Detail & Related papers  (2024-11-25T20:07:07Z)
• PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting [59.277480452459315]
  We propose a principled sensitivity pruning score that preserves visual fidelity and foreground details at significantly higher compression ratios.<n>We also propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing its training pipeline.
  arXiv  Detail & Related papers  (2024-06-14T17:53:55Z)
• R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction [53.19869886963333]
  3D Gaussian splatting (3DGS) has shown promising results in rendering image and surface reconstruction. This paper introduces R2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction.
  arXiv  Detail & Related papers  (2024-05-31T08:39:02Z)
• Instant Multi-View Head Capture through Learnable Registration [62.70443641907766]
  Existing methods for capturing datasets of 3D heads in dense semantic correspondence are slow. We introduce TEMPEH to directly infer 3D heads in dense correspondence from calibrated multi-view images. Predicting one head takes about 0.3 seconds with a median reconstruction error of 0.26 mm, 64% lower than the current state-of-the-art.
  arXiv  Detail & Related papers  (2023-06-12T21:45:18Z)
• NeRF-GAN Distillation for Efficient 3D-Aware Generation with Convolutions [97.27105725738016]
  integration of Neural Radiance Fields (NeRFs) and generative models, such as Generative Adversarial Networks (GANs) has transformed 3D-aware generation from single-view images. We propose a simple and effective method, based on re-using the well-disentangled latent space of a pre-trained NeRF-GAN in a pose-conditioned convolutional network to directly generate 3D-consistent images corresponding to the underlying 3D representations.
  arXiv  Detail & Related papers  (2023-03-22T18:59:48Z)
• Fusarium head blight detection, spikelet estimation, and severity assessment in wheat using 3D convolutional neural networks [0.0]
  Fusarium head blight (FHB) is one of the most significant diseases affecting wheat and other small grain cereals worldwide. The applications considered in this work are the automated detection of FHB disease symptoms expressed on a wheat plant, the automated estimation of the total number of spikelets and the total number of infected spikelets on a wheat head, and the automated assessment of the FHB severity in infected wheat.
  arXiv  Detail & Related papers  (2023-03-10T00:46:32Z)
• Generative models-based data labeling for deep networks regression: application to seed maturity estimation from UAV multispectral images [3.6868861317674524]
  Monitoring seed maturity is an increasing challenge in agriculture due to climate change and more restrictive practices. Traditional methods are based on limited sampling in the field and analysis in laboratory. We propose a method for estimating parsley seed maturity using multispectral UAV imagery, with a new approach for automatic data labeling.
  arXiv  Detail & Related papers  (2022-08-09T09:06:51Z)
• WheatNet: A Lightweight Convolutional Neural Network for High-throughput Image-based Wheat Head Detection and Counting [12.735055892742647]
  We propose a novel deep learning framework to accurately and efficiently count wheat heads to aid in the gathering of real-time data for decision making. We call our model WheatNet and show that our approach is robust and accurate for a wide range of environmental conditions of the wheat field. Our proposed method achieves an MAE and RMSE of 3.85 and 5.19 in our wheat head counting task, respectively, while having significantly fewer parameters when compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-17T02:38:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.