ComboNet: Combined 2D & 3D Architecture for Aorta Segmentation

• URL: http://arxiv.org/abs/2006.05325v1
• Date: Tue, 9 Jun 2020 15:02:55 GMT
• Title: ComboNet: Combined 2D & 3D Architecture for Aorta Segmentation
• Authors: Orhan Akal, Zhigang Peng and Gerardo Hermosillo Valadez
• Abstract summary: 3D segmentation with deep learning if trained with full resolution is the ideal way of achieving the best accuracy. Most of the 3D segmentation applications handle sub-sampled input instead of full resolution, which comes with the cost of losing precision at the boundary. Combonet is designed in an end to end fashion with three sub-network structures.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D segmentation with deep learning if trained with full resolution is the ideal way of achieving the best accuracy. Unlike in 2D, 3D segmentation generally does not have sparse outliers, prevents leakage to surrounding soft tissues, at the very least it is generally more consistent than 2D segmentation. However, GPU memory is generally the bottleneck for such an application. Thus, most of the 3D segmentation applications handle sub-sampled input instead of full resolution, which comes with the cost of losing precision at the boundary. In order to maintain precision at the boundary and prevent sparse outliers and leakage, we designed ComboNet. ComboNet is designed in an end to end fashion with three sub-network structures. The first two are parallel: 2D UNet with full resolution and 3D UNet with four times sub-sampled input. The last stage is the concatenation of 2D and 3D outputs along with a full-resolution input image which is followed by two convolution layers either with 2D or 3D convolutions. With ComboNet we have achieved $92.1\%$ dice accuracy for aorta segmentation. With Combonet, we have observed up to $2.3\%$ improvement of dice accuracy as opposed to 2D UNet with the full-resolution input image.

Related papers

• ConDense: Consistent 2D/3D Pre-training for Dense and Sparse Features from Multi-View Images [47.682942867405224]
  ConDense is a framework for 3D pre-training utilizing existing 2D networks and large-scale multi-view datasets. We propose a novel 2D-3D joint training scheme to extract co-embedded 2D and 3D features in an end-to-end pipeline.
  arXiv  Detail & Related papers  (2024-08-30T05:57:01Z)
• SAM-guided Graph Cut for 3D Instance Segmentation [60.75119991853605]
  This paper addresses the challenge of 3D instance segmentation by simultaneously leveraging 3D geometric and multi-view image information. We introduce a novel 3D-to-2D query framework to effectively exploit 2D segmentation models for 3D instance segmentation. Our method achieves robust segmentation performance and can generalize across different types of scenes.
  arXiv  Detail & Related papers  (2023-12-13T18:59:58Z)
• SATR: Zero-Shot Semantic Segmentation of 3D Shapes [74.08209893396271]
  We explore the task of zero-shot semantic segmentation of 3D shapes by using large-scale off-the-shelf 2D image recognition models. We develop the Assignment with Topological Reweighting (SATR) algorithm and evaluate it on ShapeNetPart and our proposed FAUST benchmarks. SATR achieves state-of-the-art performance and outperforms a baseline algorithm by 1.3% and 4% average mIoU.
  arXiv  Detail & Related papers  (2023-04-11T00:43:16Z)
• Lightweight integration of 3D features to improve 2D image segmentation [1.3799488979862027]
  We show that image segmentation can benefit from 3D geometric information without requiring a 3D groundtruth. Our method can be applied to many 2D segmentation networks, improving significantly their performance.
  arXiv  Detail & Related papers  (2022-12-16T08:22:55Z)
• Conjugate Product Graphs for Globally Optimal 2D-3D Shape Matching [12.740151710302397]
  We consider the problem of finding a continuous and non-rigid matching between a 2D contour and a 3D mesh. Existing solutions heavily rely on unrealistic prior assumptions to avoid degenerate solutions. We propose a novel 2D-3D shape matching formalism based on the conjugate product graph between the 2D contour and the 3D shape.
  arXiv  Detail & Related papers  (2022-11-21T15:41:28Z)
• SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling [75.957103837167]
  Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch.
  arXiv  Detail & Related papers  (2022-08-14T16:37:51Z)
• Homography Loss for Monocular 3D Object Detection [54.04870007473932]
  A differentiable loss function, termed as Homography Loss, is proposed to achieve the goal, which exploits both 2D and 3D information. Our method yields the best performance compared with the other state-of-the-arts by a large margin on KITTI 3D datasets.
  arXiv  Detail & Related papers  (2022-04-02T03:48:03Z)
• Multi-Modality Task Cascade for 3D Object Detection [22.131228757850373]
  Many methods train two models in isolation and use simple feature concatenation to represent 3D sensor data. We propose a novel Multi-Modality Task Cascade network (MTC-RCNN) that leverages 3D box proposals to improve 2D segmentation predictions. We show that including a 2D network between two stages of 3D modules significantly improves both 2D and 3D task performance.
  arXiv  Detail & Related papers  (2021-07-08T17:55:01Z)
• FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection [78.00922683083776]
  It is non-trivial to make a general adapted 2D detector work in this 3D task. In this technical report, we study this problem with a practice built on fully convolutional single-stage detector. Our solution achieves 1st place out of all the vision-only methods in the nuScenes 3D detection challenge of NeurIPS 2020.
  arXiv  Detail & Related papers  (2021-04-22T09:35:35Z)
• 3D-to-2D Distillation for Indoor Scene Parsing [78.36781565047656]
  We present a new approach that enables us to leverage 3D features extracted from large-scale 3D data repository to enhance 2D features extracted from RGB images. First, we distill 3D knowledge from a pretrained 3D network to supervise a 2D network to learn simulated 3D features from 2D features during the training. Second, we design a two-stage dimension normalization scheme to calibrate the 2D and 3D features for better integration. Third, we design a semantic-aware adversarial training model to extend our framework for training with unpaired 3D data.
  arXiv  Detail & Related papers  (2021-04-06T02:22:24Z)
• Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation [18.76436457395804]
  Multi-organ segmentation is one of most successful applications of deep learning in medical image analysis. Deep convolutional neural nets (CNNs) have shown great promise in achieving clinically applicable image segmentation performance on CT or MRI images. We propose a new framework for combining 3D and 2D models, in which the segmentation is realized through high-resolution 2D convolutions.
  arXiv  Detail & Related papers  (2020-12-16T21:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.