Probabilistic U-Net with Kendall Shape Spaces for Geometry-Aware Segmentations of Images

• URL: http://arxiv.org/abs/2410.14017v1
• Date: Thu, 17 Oct 2024 20:32:43 GMT
• Title: Probabilistic U-Net with Kendall Shape Spaces for Geometry-Aware Segmentations of Images
• Authors: Jiyoung Park, Günay Doğan,
• Abstract summary: We propose a probabilistic image segmentation model that can incorporate the geometry of a segmentation. Our model also adopts the Kendall Shape Variational Auto-Encoder of citevadgama2023kendall to encode a Kendall shape space.
• Score: 0.3499042782396683
• License:
• Abstract: One of the fundamental problems in computer vision is image segmentation, the task of detecting distinct regions or objects in given images. Deep Neural Networks (DNN) have been shown to be very effective in segmenting challenging images, producing convincing segmentations. There is further need for probabilistic DNNs that can reflect the uncertainties from the input images and the models into the computed segmentations, in other words, new DNNs that can generate multiple plausible segmentations and their distributions depending on the input or the model uncertainties. While there are existing probabilistic segmentation models, many of them do not take into account the geometry or shape underlying the segmented regions. In this paper, we propose a probabilistic image segmentation model that can incorporate the geometry of a segmentation. Our proposed model builds on the Probabilistic U-Net of \cite{kohl2018probabilistic} to generate probabilistic segmentations, i.e.\! multiple likely segmentations for an input image. Our model also adopts the Kendall Shape Variational Auto-Encoder of \cite{vadgama2023kendall} to encode a Kendall shape space in the latent variable layers of the prior and posterior networks of the Probabilistic U-Net. Incorporating the shape space in this manner leads to a more robust segmentation with spatially coherent regions, respecting the underlying geometry in the input images.

Related papers

• View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
  We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene. Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output. We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
  arXiv  Detail & Related papers  (2024-05-30T04:14:58Z)
• Variational multichannel multiclass segmentation using unsupervised lifting with CNNs [0.0]
  We implement a flexible multiclass segmentation method that divides a given image into $K$ different regions. We use convolutional neural networks (CNNs) targeting a pre-decomposition of the image. Special emphasis is given to the extraction of informative feature maps serving as a starting point for the segmentation.
  arXiv  Detail & Related papers  (2023-02-04T18:01:47Z)
• Towards General-Purpose Representation Learning of Polygonal Geometries [62.34832826705641]
  We develop a general-purpose polygon encoding model, which can encode a polygonal geometry into an embedding space. We conduct experiments on two tasks: 1) shape classification based on MNIST; 2) spatial relation prediction based on two new datasets - DBSR-46K and DBSR-cplx46K. Our results show that NUFTspec and ResNet1D outperform multiple existing baselines with significant margins.
  arXiv  Detail & Related papers  (2022-09-29T15:59:23Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• A singular Riemannian geometry approach to Deep Neural Networks II. Reconstruction of 1-D equivalence classes [78.120734120667]
  We build the preimage of a point in the output manifold in the input space. We focus for simplicity on the case of neural networks maps from n-dimensional real spaces to (n - 1)-dimensional real spaces.
  arXiv  Detail & Related papers  (2021-12-17T11:47:45Z)
• Contour Proposal Networks for Biomedical Instance Segmentation [0.8602553195689513]
  We present a conceptually simple framework for object instance segmentation called Contour Proposal Network (CPN) CPN detects possibly overlapping objects in an image while simultaneously fitting closed object contours using an interpretable, fixed-sized representation based on Fourier Descriptors. We show CPNs that outperform U-Nets and Mask R-CNNs in instance segmentation accuracy and present variants with execution times suitable for real-time applications.
  arXiv  Detail & Related papers  (2021-04-07T21:00:45Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty [32.33791302617957]
  We introduce segmentation networks (SSNs), an efficient probabilistic method for modelling aleatoric uncertainty with any image segmentation network architecture. SSNs can generate multiple spatially coherent hypotheses for a single image. We tested our method on the segmentation of real-world medical data, including lung nodules in 2D CT and brain tumours in 3D multimodal MRI scans.
  arXiv  Detail & Related papers  (2020-06-10T18:06:41Z)
• CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-24T04:55:43Z)
• Deep Convolutional Neural Networks with Spatial Regularization, Volume and Star-shape Priori for Image Segmentation [6.282154392910916]
  The classification functions in the existing network architecture of CNNs are simple and lack capabilities to handle important spatial information. We propose a novel Soft Threshold Dynamics (STD) framework which can easily integrate many spatial priors into the DCNNs. The proposed method is a general mathematical framework and it can be applied to any semantic segmentation DCNNs.
  arXiv  Detail & Related papers  (2020-02-10T18:03:44Z)
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey [0.0]
  We take a glance at the evolution of both semantic and instance segmentation work based on CNN. We have given a glimpse of some state-of-the-art panoptic segmentation models.
  arXiv  Detail & Related papers  (2020-01-13T06:07:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.