PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage

• URL: http://arxiv.org/abs/2206.13082v3
• Date: Sat, 20 Jan 2024 01:23:32 GMT
• Title: PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage
• Authors: Ruiming Du, Zhihong Ma, Pengyao Xie, Yong He, Haiyan Cen
• Abstract summary: deep learning network plant segmentation transformer (PST) PST is composed of: (i) a dynamic voxel feature encoder (DVFE) to aggregate the point features with the raw spatial resolution; (ii) a dual window sets attention blocks to capture contextual information; and (iii) a dense feature propagation module to obtain the final dense point feature map. Results: PST and PST-PointGroup (PG) achieved superior performance in semantic and instance segmentation tasks.
• Score: 5.010317705589445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmentation of plant point clouds to obtain high-precise morphological traits is essential for plant phenotyping. Although the fast development of deep learning has boosted much research on segmentation of plant point clouds, previous studies mainly focus on the hard voxelization-based or down-sampling-based methods, which are limited to segmenting simple plant organs. Segmentation of complex plant point clouds with a high spatial resolution still remains challenging. In this study, we proposed a deep learning network plant segmentation transformer (PST) to achieve the semantic and instance segmentation of rapeseed plants point clouds acquired by handheld laser scanning (HLS) with the high spatial resolution, which can characterize the tiny siliques as the main traits targeted. PST is composed of: (i) a dynamic voxel feature encoder (DVFE) to aggregate the point features with the raw spatial resolution; (ii) the dual window sets attention blocks to capture the contextual information; and (iii) a dense feature propagation module to obtain the final dense point feature map. The results proved that PST and PST-PointGroup (PG) achieved superior performance in semantic and instance segmentation tasks. For the semantic segmentation, the mean IoU, mean Precision, mean Recall, mean F1-score, and overall accuracy of PST were 93.96%, 97.29%, 96.52%, 96.88%, and 97.07%, achieving an improvement of 7.62%, 3.28%, 4.8%, 4.25%, and 3.88% compared to the second-best state-of-the-art network PAConv. For instance segmentation, PST-PG reached 89.51%, 89.85%, 88.83% and 82.53% in mCov, mWCov, mPerc90, and mRec90, achieving an improvement of 2.93%, 2.21%, 1.99%, and 5.9% compared to the original PG. This study proves that the deep-learning-based point cloud segmentation method has a great potential for resolving dense plant point clouds with complex morphological traits.

Related papers

• Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning [49.91297276176978]
  We propose a novel. Efficient Fine-Tuning (PEFT) method for point cloud, called Point GST. Point GST freezes the pre-trained model and introduces a trainable Point Cloud Spectral Adapter (PCSA) to finetune parameters in the spectral domain. Extensive experiments on challenging point cloud datasets demonstrate that Point GST not only outperforms its fully finetuning counterpart but also significantly reduces trainable parameters.
  arXiv  Detail & Related papers  (2024-10-10T17:00:04Z)
• Point Cloud Pre-training with Diffusion Models [62.12279263217138]
  We propose a novel pre-training method called Point cloud Diffusion pre-training (PointDif) PointDif achieves substantial improvement across various real-world datasets for diverse downstream tasks such as classification, segmentation and detection.
  arXiv  Detail & Related papers  (2023-11-25T08:10:05Z)
• PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds [99.60575439926963]
  We propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings. PointHPS iteratively refines point features through a cascaded architecture. Extensive experiments demonstrate that PointHPS, with its powerful point feature extraction and processing scheme, outperforms State-of-the-Art methods.
  arXiv  Detail & Related papers  (2023-08-28T11:10:14Z)
• Clustering based Point Cloud Representation Learning for 3D Analysis [80.88995099442374]
  We propose a clustering based supervised learning scheme for point cloud analysis. Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space. Our algorithm shows notable improvements on famous point cloud segmentation datasets.
  arXiv  Detail & Related papers  (2023-07-27T03:42:12Z)
• Benchmarking Deep Learning Architectures for Urban Vegetation Point Cloud Semantic Segmentation from MLS [0.5999777817331315]
  Vegetation is crucial for sustainable and resilient cities providing ecosystem services and well-being of humans. Recently, deep learning for point cloud semantic segmentation has shown significant progress. We provide an assessment of point-based deep learning models for semantic segmentation of vegetation class.
  arXiv  Detail & Related papers  (2023-06-17T06:38:42Z)
• PlantDet: A benchmark for Plant Detection in the Three-Rivers-Source Region [4.676030127116814]
  We construct a dataset for Plant detection in the Three-River-Source region (PTRS) It comprises 21 types, 6965 high-resolution images of 2160*3840 pixels, captured by diverse sensors and platforms, and featuring objects of varying shapes and sizes. The PTRS presents us with challenges such as dense occlusion, varying leaf resolutions, and high feature similarity among plants, prompting us to develop a novel object detection network named PlantDet.
  arXiv  Detail & Related papers  (2023-04-11T04:18:56Z)
• Spatial Layout Consistency for 3D Semantic Segmentation [0.7614628596146599]
  We introduce a novel deep convolutional neural network (DCNN) technique for achieving voxel-based semantic segmentation of the ALTM's point clouds. The suggested deep learning method, Semantic Utility Network (SUNet) is a multi-dimensional and multi-resolution network. Our experiments demonstrated that SUNet's spatial layout consistency and a multi-resolution feature aggregation could significantly improve performance.
  arXiv  Detail & Related papers  (2023-03-02T03:24:21Z)
• Eff-3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient point clouds [1.5882586857953638]
  We propose a novel weakly supervised framework, Eff-3DPSeg, for 3D plant shoot segmentation. High-resolution point clouds of soybean were reconstructed using a low-cost photogrammetry system. A weakly-supervised deep learning method was proposed for plant organ segmentation.
  arXiv  Detail & Related papers  (2022-12-20T14:09:37Z)
• A Deep Learning Approach Based on Graphs to Detect Plantation Lines [16.76043873454695]
  We propose a deep learning approach based on graphs to detect plantation lines in UAV-based RGB imagery. The proposed method was compared against state-of-the-art deep learning methods. It achieved superior performance with a significant margin, returning precision, recall, and F1-score of 98.7%, 91.9%, and 95.1%, respectively.
  arXiv  Detail & Related papers  (2021-02-05T14:56:42Z)
• Point Transformer [122.2917213154675]
  We investigate the application of self-attention networks to 3D point cloud processing. We design self-attention layers for point clouds and use these to construct self-attention networks for tasks such as semantic scene segmentation. Our Point Transformer design improves upon prior work across domains and tasks.
  arXiv  Detail & Related papers  (2020-12-16T18:58:56Z)
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation [111.7241018610573]
  We present PointGroup, a new end-to-end bottom-up architecture for instance segmentation. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best
  arXiv  Detail & Related papers  (2020-04-03T16:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.