PlantSegNeRF: A few-shot, cross-dataset method for plant 3D instance point cloud reconstruction via joint-channel NeRF with multi-view image instance matching

• URL: http://arxiv.org/abs/2507.00371v1
• Date: Tue, 01 Jul 2025 01:59:59 GMT
• Title: PlantSegNeRF: A few-shot, cross-dataset method for plant 3D instance point cloud reconstruction via joint-channel NeRF with multi-view image instance matching
• Authors: Xin Yang, Ruiming Du, Hanyang Huang, Jiayang Xie, Pengyao Xie, Leisen Fang, Ziyue Guo, Nanjun Jiang, Yu Jiang, Haiyan Cen,
• Abstract summary: We propose a novel approach called plant segmentation neural radiance fields (PlantSegNeRF) to generate high-precision instance point clouds.<n>PlantSegNeRF performed 2D instance segmentation on the multi-view images to generate instance masks for each organ with a corresponding ID.<n>It achieved average improvements of 11.7%, 38.2%, 32.2% and 25.3% in mPrec, mRec, mCov, mWCov, respectively.
• Score: 6.041945288896862
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Organ segmentation of plant point clouds is a prerequisite for the high-resolution and accurate extraction of organ-level phenotypic traits. Although the fast development of deep learning has boosted much research on segmentation of plant point clouds, the existing techniques for organ segmentation still face limitations in resolution, segmentation accuracy, and generalizability across various plant species. In this study, we proposed a novel approach called plant segmentation neural radiance fields (PlantSegNeRF), aiming to directly generate high-precision instance point clouds from multi-view RGB image sequences for a wide range of plant species. PlantSegNeRF performed 2D instance segmentation on the multi-view images to generate instance masks for each organ with a corresponding ID. The multi-view instance IDs corresponding to the same plant organ were then matched and refined using a specially designed instance matching module. The instance NeRF was developed to render an implicit scene, containing color, density, semantic and instance information. The implicit scene was ultimately converted into high-precision plant instance point clouds based on the volume density. The results proved that in semantic segmentation of point clouds, PlantSegNeRF outperformed the commonly used methods, demonstrating an average improvement of 16.1%, 18.3%, 17.8%, and 24.2% in precision, recall, F1-score, and IoU compared to the second-best results on structurally complex datasets. More importantly, PlantSegNeRF exhibited significant advantages in plant point cloud instance segmentation tasks. Across all plant datasets, it achieved average improvements of 11.7%, 38.2%, 32.2% and 25.3% in mPrec, mRec, mCov, mWCov, respectively. This study extends the organ-level plant phenotyping and provides a high-throughput way to supply high-quality 3D data for the development of large-scale models in plant science.

Related papers

• Graph-Based Deep Learning for Component Segmentation of Maize Plants [0.0]
  We propose a novel Deep Learning architecture to detect components of individual plants on LiDAR 3D Point Cloud data sets.<n>This architecture is based on the concept of Graph Neural Networks (GNN) and feature enhancing with Principal Component Analysis (PCA)<n>Our graph-based deep learning approach enhances segmentation accuracy for identifying individual plant components, achieving percentages above 80% in the IoU average.
  arXiv  Detail & Related papers  (2025-06-30T18:44:27Z)
• Wheat3DGS: In-field 3D Reconstruction, Instance Segmentation and Phenotyping of Wheat Heads with Gaussian Splatting [1.4100451538155885]
  We present Wheat3DGS, a novel approach that leverages 3DGS and the Segment Anything Model (SAM) for precise 3D instance segmentation and morphological measurement of hundreds of wheat heads automatically.<n>We validate the accuracy of wheat breeding head extraction against high-resolution laser scan data, obtaining per-instance mean absolute percentage errors of 15.1%, 18.3%, and 40.2% for length, width, and volume.
  arXiv  Detail & Related papers  (2025-04-09T15:31:42Z)
• PCE-GAN: A Generative Adversarial Network for Point Cloud Attribute Quality Enhancement based on Optimal Transport [56.56430888985025]
  We propose a generative adversarial network for point cloud quality enhancement (PCE-GAN)<n>The generator consists of a local feature extraction (LFE) unit, a global spatial correlation (GSC) unit and a feature squeeze unit.<n>The discriminator computes the deviation between the probability distributions of the enhanced point cloud and the original point cloud, guiding the generator to achieve high quality reconstruction.
  arXiv  Detail & Related papers  (2025-02-26T07:34:33Z)
• Diffusion-based Data Augmentation for Nuclei Image Segmentation [68.28350341833526]
  We introduce the first diffusion-based augmentation method for nuclei segmentation. The idea is to synthesize a large number of labeled images to facilitate training the segmentation model. The experimental results show that by augmenting 10% labeled real dataset with synthetic samples, one can achieve comparable segmentation results.
  arXiv  Detail & Related papers  (2023-10-22T06:16:16Z)
• PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds [99.60575439926963]
  We propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings. PointHPS iteratively refines point features through a cascaded architecture. Extensive experiments demonstrate that PointHPS, with its powerful point feature extraction and processing scheme, outperforms State-of-the-Art methods.
  arXiv  Detail & Related papers  (2023-08-28T11:10:14Z)
• Eff-3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient point clouds [1.5882586857953638]
  We propose a novel weakly supervised framework, Eff-3DPSeg, for 3D plant shoot segmentation. High-resolution point clouds of soybean were reconstructed using a low-cost photogrammetry system. A weakly-supervised deep learning method was proposed for plant organ segmentation.
  arXiv  Detail & Related papers  (2022-12-20T14:09:37Z)
• CASSPR: Cross Attention Single Scan Place Recognition [43.68230981047338]
  Place recognition based on point clouds (LiDAR) is an important component for autonomous robots or self-driving vehicles. Current SOTA performance is achieved on accumulated LiDAR submaps using either point-based or voxel-based structures. We propose CASSPR as a method to fuse point-based and voxel-based approaches using cross attention transformers.
  arXiv  Detail & Related papers  (2022-11-22T19:18:30Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage [5.010317705589445]
  deep learning network plant segmentation transformer (PST) PST is composed of: (i) a dynamic voxel feature encoder (DVFE) to aggregate the point features with the raw spatial resolution; (ii) a dual window sets attention blocks to capture contextual information; and (iii) a dense feature propagation module to obtain the final dense point feature map. Results: PST and PST-PointGroup (PG) achieved superior performance in semantic and instance segmentation tasks.
  arXiv  Detail & Related papers  (2022-06-27T06:56:48Z)
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution [136.7261709896713]
  We propose a data-driven approach that generates the appropriate convolution kernels to apply in response to the nature of the instances. The proposed method achieves promising results on both ScanetNetV2 and S3DIS. It also improves inference speed by more than 25% over the current state-of-the-art.
  arXiv  Detail & Related papers  (2020-11-26T14:56:57Z)
• Stem-leaf segmentation and phenotypic trait extraction of maize shoots from three-dimensional point cloud [8.392251372468412]
  We propose an automatic stem-leaf segmentation method consisting of three main steps: skeleton extraction, coarse segmentation based on the skeleton, fine segmentation based on stem-leaf classification. Six phenotypic parameters can be accurately and automatically measured, including plant height, crown diameter, stem height and diameter, leaf width and length. The proposed algorithm could precisely segment not only the fully expanded leaves, but also the new leaves wrapped together and close together.
  arXiv  Detail & Related papers  (2020-09-07T13:58:09Z)
• Two-View Fine-grained Classification of Plant Species [66.75915278733197]
  We propose a novel method based on a two-view leaf image representation and a hierarchical classification strategy for fine-grained recognition of plant species. A deep metric based on Siamese convolutional neural networks is used to reduce the dependence on a large number of training samples and make the method scalable to new plant species.
  arXiv  Detail & Related papers  (2020-05-18T21:57:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.