Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs

• URL: http://arxiv.org/abs/2609.07724v1
• Date: Mon, 07 Sep 2026 16:35:01 GMT
• Title: Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs
• Abstract summary: Our pipeline produces semantically labeled 3D point clouds without manual annotation, per-species fine-tuning, or domain-specific preprocessing.<n>On a controlled Begonia maculata testbed the SAM3 pipeline achieves 92.6% mIoU, reaching 95.9% of the oracle upper bound established with perfect ground-truth masks.
• Score: 0.9176056742068813
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate 3D plant organ segmentation is fundamental to automated phenotyping. Existing approaches rely on annotated training data or species-specific model configurations. We present an annotation-free pipeline for 3D plant organ segmentation, combining text-prompted SAM3 segmentation with semantic neural radiance fields (NeRFs). Given only multi-view RGB images and a list of class names, our zero-shot pipeline produces semantically labeled 3D point clouds without manual annotation, per-species fine-tuning, or domain-specific preprocessing. Multi-view NeRF fusion acts as effective implicit consensus mechanism that lifts imperfect per-frame masks into accurate 3D labels. On a controlled Begonia maculata testbed the SAM3 pipeline achieves 92.6% mIoU, reaching 95.9% of the oracle upper bound established with perfect ground-truth masks. The pipeline was further evaluated on a new dataset spanning ten diverse plant point clouds reaching an average 0.856 mIoU, with leaf and pot IoU above 0.91 and 0.90 for every species, respectively. These results demonstrate that annotation-free 3D plant organ segmentation is now feasible and approaching the range of supervised methods.

Related papers

• Native space based pipelines outperform template space based pipeline in subcortical segmentation [0.0]
  We evaluate whether native space approaches offer a measurable advantage.<n>Native-space segmentation is preferable for applications requiring patient specific anatomical fidelity.<n> bridging the 7T-to-3T domain gap remains an open challenge.
  arXiv  Detail & Related papers  (2026-06-19T14:18:31Z)
• OmniPlantSeg: Species Agnostic 3D Point Cloud Organ Segmentation for High-Resolution Plant Phenotyping Across Modalities [4.770378899761483]
  It is common to use extensive pre-processing and down-sample the plant point clouds to meet hardware or neural network input size requirements.<n>We propose a simple, yet effective algorithm KDSS for sub-sampling of biological point clouds that is agnostic to sensor data and plant species.
  arXiv  Detail & Related papers  (2025-09-25T11:45:14Z)
• MaizeField3D: A Curated 3D Point Cloud and Procedural Model Dataset of Field-Grown Maize from a Diversity Panel [12.89812013060155]
  We present a curated dataset of 3D point clouds of field-grown maize plants from a diverse genetic panel.<n>MaizeField3D will serve as a foundational dataset for AI-driven phenotyping, plant structural analysis, and 3D applications in agricultural research.
  arXiv  Detail & Related papers  (2025-03-10T19:53:20Z)
• Diffusion-SS3D: Diffusion Model for Semi-supervised 3D Object Detection [77.23918785277404]
  We present Diffusion-SS3D, a new perspective of enhancing the quality of pseudo-labels via the diffusion model for semi-supervised 3D object detection. Specifically, we include noises to produce corrupted 3D object size and class label, distributions, and then utilize the diffusion model as a denoising process to obtain bounding box outputs. We conduct experiments on the ScanNet and SUN RGB-D benchmark datasets to demonstrate that our approach achieves state-of-the-art performance against existing methods.
  arXiv  Detail & Related papers  (2023-12-05T18:54:03Z)
• Eff-3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient point clouds [1.5882586857953638]
  We propose a novel weakly supervised framework, Eff-3DPSeg, for 3D plant shoot segmentation. High-resolution point clouds of soybean were reconstructed using a low-cost photogrammetry system. A weakly-supervised deep learning method was proposed for plant organ segmentation.
  arXiv  Detail & Related papers  (2022-12-20T14:09:37Z)
• SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloud [125.9472454212909]
  We present a novel Semi-Supervised Domain Adaptation method for 3D object detection (SSDA3D) SSDA3D includes an Inter-domain Adaptation stage and an Intra-domain Generalization stage. Experiments show that, with only 10% labeled target data, our SSDA3D can surpass the fully-supervised oracle model with 100% target label.
  arXiv  Detail & Related papers  (2022-12-06T09:32:44Z)
• Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
  We train a semantic point cloud segmentation network with only a small portion of points being labeled. We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples. Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
  arXiv  Detail & Related papers  (2021-07-23T14:34:57Z)
• Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction [57.74609918453932]
  Automatic blood vessel extraction from 3D medical images is crucial for vascular disease diagnoses. Existing methods may suffer from discontinuities of extracted vessels when segmenting such thin tubular structures from 3D images. We argue that preserving the continuity of extracted vessels requires to take into account the global geometry. We propose a hybrid representation learning approach to address this challenge.
  arXiv  Detail & Related papers  (2020-12-14T05:22:49Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation [81.02742110604161]
  State-of-the-art methods for large-scale driving-scene LiDAR segmentation often project the point clouds to 2D space and then process them via 2D convolution. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pat-tern. Our method achieves the 1st place in the leaderboard of Semantic KITTI and outperforms existing methods on nuScenes with a noticeable margin, about 4%.
  arXiv  Detail & Related papers  (2020-11-19T18:53:11Z)
• Stem-leaf segmentation and phenotypic trait extraction of maize shoots from three-dimensional point cloud [8.392251372468412]
  We propose an automatic stem-leaf segmentation method consisting of three main steps: skeleton extraction, coarse segmentation based on the skeleton, fine segmentation based on stem-leaf classification. Six phenotypic parameters can be accurately and automatically measured, including plant height, crown diameter, stem height and diameter, leaf width and length. The proposed algorithm could precisely segment not only the fully expanded leaves, but also the new leaves wrapped together and close together.
  arXiv  Detail & Related papers  (2020-09-07T13:58:09Z)
• Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds [49.47017280475232]
  We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds. We take a Complete and Label approach to recover the underlying surfaces before passing them to a segmentation network. The recovered 3D surfaces serve as a canonical domain, from which semantic labels can transfer across different LiDAR sensors.
  arXiv  Detail & Related papers  (2020-07-16T17:42:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.