FreSeg: Frenet-Frame-based Part Segmentation for 3D Curvilinear Structures

• URL: http://arxiv.org/abs/2404.14435v1
• Date: Fri, 19 Apr 2024 16:40:24 GMT
• Title: FreSeg: Frenet-Frame-based Part Segmentation for 3D Curvilinear Structures
• Authors: Shixuan Gu, Jason Ken Adhinarta, Mikhail Bessmeltsev, Jiancheng Yang, Jessica Zhang, Daniel Berger, Jeff W. Lichtman, Hanspeter Pfister, Donglai Wei,
• Abstract summary: Part segmentation is a crucial task for 3D curvilinear structures like neuron dendrites and blood vessels. We propose FreSeg, a framework of part segmentation tasks for 3D curvilinear structures.
• Score: 26.814165975617136
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Part segmentation is a crucial task for 3D curvilinear structures like neuron dendrites and blood vessels, enabling the analysis of dendritic spines and aneurysms with scientific and clinical significance. However, their diversely winded morphology poses a generalization challenge to existing deep learning methods, which leads to labor-intensive manual correction. In this work, we propose FreSeg, a framework of part segmentation tasks for 3D curvilinear structures. With Frenet-Frame-based point cloud transformation, it enables the models to learn more generalizable features and have significant performance improvements on tasks involving elongated and curvy geometries. We evaluate FreSeg on 2 datasets: 1) DenSpineEM, an in-house dataset for dendritic spine segmentation, and 2) IntrA, a public 3D dataset for intracranial aneurysm segmentation. Further, we will release the DenSpineEM dataset, which includes roughly 6,000 spines from 69 dendrites from 3 public electron microscopy (EM) datasets, to foster the development of effective dendritic spine instance extraction methods and, consequently, large-scale connectivity analysis to better understand mammalian brains.

Related papers

• trAIce3D: A Prompt-Driven Transformer Based U-Net for Semantic Segmentation of Microglial Cells from Large-Scale 3D Microscopy Images [39.58317527488534]
  We introduce trAIce3D, a deep-learning architecture designed for precise microglia segmentation.<n>It employs a two-stage approach: first, a 3D U-Net with vision transformers in the encoder detects somas using a sliding-window technique to cover the entire image.<n>It then refines each soma and its branches by using soma coordinates as a prompt and a 3D window around the target cell as input.<n>trained and evaluated on a dataset of 41,230 microglial cells, trAIce3D significantly improves segmentation accuracy and scalable generalization.
  arXiv  Detail & Related papers  (2025-07-30T12:54:53Z)
• Geometric Framework for 3D Cell Segmentation Correction [3.540684770290861]
  3D cellular image segmentation methods are commonly divided into non-2D and 2D-based approaches. Errors in 2D results often propagate, leading to oversegmentations in the final 3D results. We introduce an interpretable geometric framework that addresses the oversegmentations by correcting the 2D segmentation results based on geometric information from adjacent layers.
  arXiv  Detail & Related papers  (2025-02-03T23:47:45Z)
• SpineMamba: Enhancing 3D Spinal Segmentation in Clinical Imaging through Residual Visual Mamba Layers and Shape Priors [10.431439196002842]
  We introduce a residual visual Mamba layer to model the deep semantic features and long-range spatial dependencies of 3D spinal data. We also propose a novel spinal shape prior module that captures specific anatomical information of the spine from medical images. SpineMamba achieves superior segmentation performance, exceeding it by up to 2 percentage points.
  arXiv  Detail & Related papers  (2024-08-28T15:59:40Z)
• ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining [104.34751911174196]
  We build a large-scale dataset of 3DGS using ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories. We introduce textbftextitGaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters.
  arXiv  Detail & Related papers  (2024-08-20T14:49:14Z)
• Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction [1.1748284119769041]
  We present an extensive evaluation of 8 2D-3D models on equal footing using 6 public datasets. Our results show that attention-based methods that capture global spatial relationships tend to perform better across all anatomies.
  arXiv  Detail & Related papers  (2023-09-24T09:05:35Z)
• NEAT: Distilling 3D Wireframes from Neural Attraction Fields [52.90572335390092]
  This paper studies the problem of structured lineframe junctions using 3D reconstruction segments andFocusing junctions. ProjectNEAT enjoys the joint neural fields and view without crossart matching from scratch.
  arXiv  Detail & Related papers  (2023-07-14T07:25:47Z)
• A geometry-aware deep network for depth estimation in monocular endoscopy [17.425158094539462]
  The proposed method is extensively validated across different datasets and clinical images. The generalizability of the proposed method achieves mean RMSE values of 12.604 (T1-L1), 9.930 (T2-L2), and 13.893 (colon) on the ColonDepth dataset.
  arXiv  Detail & Related papers  (2023-04-20T11:59:32Z)
• Semi-Supervised Hybrid Spine Network for Segmentation of Spine MR Images [14.190504802866288]
  We propose a two-stage algorithm, named semi-supervised hybrid spine network (SSHSNet) to achieve simultaneous vertebral bodies (VBs) and intervertebral discs (IVDs) segmentation. In the first stage, we constructed a 2D semi-supervised DeepLabv3+ by using cross pseudo supervision to obtain intra-slice features and coarse segmentation. In the second stage, a 3D full-resolution patch-based DeepLabv3+ was built to extract inter-slice information. Results show that the proposed method has great potential in dealing with the data imbalance problem
  arXiv  Detail & Related papers  (2022-03-23T02:57:14Z)
• Universal Segmentation of 33 Anatomies [19.194539991903593]
  We present an approach for learning a single model that universally segments 33 anatomical structures. We learn such a model from a union of multiple datasets, with each dataset containing the images that are partially labeled. We evaluate our model on multiple open-source datasets, proving that our model has a good generalization performance.
  arXiv  Detail & Related papers  (2022-03-04T02:29:54Z)
• TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
  We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes. We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
  arXiv  Detail & Related papers  (2020-12-26T08:02:56Z)
• Mix Dimension in Poincar\'{e} Geometry for 3D Skeleton-based Action Recognition [57.98278794950759]
  Graph Convolutional Networks (GCNs) have already demonstrated their powerful ability to model the irregular data. We present a novel spatial-temporal GCN architecture which is defined via the Poincar'e geometry. We evaluate our method on two current largest scale 3D datasets.
  arXiv  Detail & Related papers  (2020-07-30T18:23:18Z)
• MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency [72.82534577726334]
  We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. Our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
  arXiv  Detail & Related papers  (2020-06-22T08:50:09Z)
• clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation [57.20783326661043]
  We introduce a novel similarity measure termed centerlineDice (short clDice) We theoretically prove that clDice guarantees topology preservation up to homotopy equivalence for binary 2D and 3D segmentation. We benchmark the soft-clDice loss on five public datasets, including vessels, roads and neurons (2D and 3D)
  arXiv  Detail & Related papers  (2020-03-16T16:27:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.