Atlas-ISTN: Joint Segmentation, Registration and Atlas Construction with Image-and-Spatial Transformer Networks

• URL: http://arxiv.org/abs/2012.10533v1
• Date: Fri, 18 Dec 2020 21:53:09 GMT
• Title: Atlas-ISTN: Joint Segmentation, Registration and Atlas Construction with Image-and-Spatial Transformer Networks
• Authors: Matthew Sinclair and Andreas Schuh and Karl Hahn and Kersten Petersen and Ying Bai and James Batten and Michiel Schaap and Ben Glocker
• Abstract summary: We propose Atlas-ISTN, a framework that jointly learns segmentation and registration on 2D and 3D image data. Atlas-ISTN learns to segment multiple structures of interest and to register the constructed, topologically consistent atlas labelmap to an intermediate pixel-wise segmentation. This process both mitigates for noise in the target image that can result in spurious pixel-wise predictions, as well as improves upon the one-pass prediction of the model.
• Score: 11.677800377183972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning models for semantic segmentation are able to learn powerful representations for pixel-wise predictions, but are sensitive to noise at test time and do not guarantee a plausible topology. Image registration models on the other hand are able to warp known topologies to target images as a means of segmentation, but typically require large amounts of training data, and have not widely been benchmarked against pixel-wise segmentation models. We propose Atlas-ISTN, a framework that jointly learns segmentation and registration on 2D and 3D image data, and constructs a population-derived atlas in the process. Atlas-ISTN learns to segment multiple structures of interest and to register the constructed, topologically consistent atlas labelmap to an intermediate pixel-wise segmentation. Additionally, Atlas-ISTN allows for test time refinement of the model's parameters to optimize the alignment of the atlas labelmap to an intermediate pixel-wise segmentation. This process both mitigates for noise in the target image that can result in spurious pixel-wise predictions, as well as improves upon the one-pass prediction of the model. Benefits of the Atlas-ISTN framework are demonstrated qualitatively and quantitatively on 2D synthetic data and 3D cardiac computed tomography and brain magnetic resonance image data, out-performing both segmentation and registration baseline models. Atlas-ISTN also provides inter-subject correspondence of the structures of interest, enabling population-level shape and motion analysis.

Related papers

• In search of truth: Evaluating concordance of AI-based anatomy segmentation models [3.740726797046942]
  AI-based methods for anatomy segmentation can help automate characterization of large imaging datasets.<n>We introduce a practical framework to assist in evaluating them on datasets that do not contain ground truth annotations.
  arXiv  Detail & Related papers  (2025-12-17T19:33:56Z)
• An Efficient Model-Driven Groupwise Approach for Atlas Construction [40.43130112593729]
  We introduce DARC (Diffeomorphic Atlas Registration via Coordinate descent), a novel model-driven groupwise registration framework for atlas construction.<n>DARC supports a broad range of image dissimilarity metrics and efficiently handles arbitrary numbers of 3D images without incurring GPU memory issues.<n>We demonstrate two key applications: (1) One-shot segmentation, where labels annotated only on the atlas are propagated to subjects via inverse deformations; and (2) shape synthesis, where new anatomical variants are generated by warping the atlas mesh.
  arXiv  Detail & Related papers  (2025-08-14T15:28:09Z)
• Adaptive Noise-Tolerant Network for Image Segmentation [1.57731592348751]
  We study whether integrating imperfect or noisy segmentation results from off-the-shelf segmentation algorithms may help achieve better segmentation results through a new Adaptive Noise-Tolerant Network (ANTN) model. We extend the noisy label deep learning to image segmentation with two novel aspects: (1) multiple noisy labels can be integrated into one deep learning model; (2) noisy segmentation modeling, including probabilistic parameters, is adaptive, depending on the given testing image appearance.
  arXiv  Detail & Related papers  (2025-01-13T09:49:34Z)
• CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations [90.50864830038202]
  We present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images. We use a dual-encoder to separately encode the images and their corresponding geo-locations, and use contrastive objectives to learn effective location representations from images. CSP significantly boosts the model performance with 10-34% relative improvement with various labeled training data sampling ratios.
  arXiv  Detail & Related papers  (2023-05-01T23:11:18Z)
• Spatial Correspondence between Graph Neural Network-Segmented Images [1.807691213023136]
  Graph neural networks (GNNs) have been proposed for medical image segmentation. This work explores the potentials in these GNNs with common topology for establishing spatial correspondence. With an example application of registering local vertebral sub-regions found in CT images, our experimental results showed that the GNN-based segmentation is capable of accurate and reliable localization.
  arXiv  Detail & Related papers  (2023-03-12T03:25:01Z)
• Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training of Image Segmentation Models [54.49581189337848]
  We propose a method to enable the end-to-end pre-training for image segmentation models based on classification datasets. The proposed method leverages a weighted segmentation learning procedure to pre-train the segmentation network en masse. Experiment results show that, with ImageNet accompanied by PSSL as the source dataset, the proposed end-to-end pre-training strategy successfully boosts the performance of various segmentation models.
  arXiv  Detail & Related papers  (2022-07-04T13:02:32Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Aladdin: Joint Atlas Building and Diffeomorphic Registration Learning with Pairwise Alignment [18.338563869053065]
  This work explores using a convolutional neural network (CNN) to jointly predict the atlas and a stationary velocity field (SVF) parameterization for diffeomorphic image registration. Our results show that the proposed framework achieves better performance than other state-of-the-art image registration algorithms.
  arXiv  Detail & Related papers  (2022-02-07T23:52:21Z)
• A Topology-Attention ConvLSTM Network and Its Application to EM Images [11.081936935096873]
  We propose a novel TopologyAttention ConvLSTM Network (TACNet) for 3D image segmentation. Specifically, we propose a Spatial Topology-Attention (STA) module to process a 3D image as a stack of 2D image slices. In order to effectively transfer topology-critical information across slices, we propose an Iterative-Topology Attention (ITA) module.
  arXiv  Detail & Related papers  (2022-02-07T01:33:01Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Deep Active Learning for Joint Classification & Segmentation with Weak Annotator [22.271760669551817]
  CNN visualization and interpretation methods, like class-activation maps (CAMs), are typically used to highlight the image regions linked to class predictions. We propose an active learning framework, which progressively integrates pixel-level annotations during training. Our results indicate that, by simply using random sample selection, the proposed approach can significantly outperform state-of-the-art CAMs and AL methods.
  arXiv  Detail & Related papers  (2020-10-10T03:25:54Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)
• Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning [86.45526827323954]
  Weakly-supervised semantic segmentation is a challenging task as no pixel-wise label information is provided for training. We propose an iterative algorithm to learn such pairwise relations. We show that the proposed algorithm performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-19T10:32:03Z)
• Self-Learning AI Framework for Skin Lesion Image Segmentation and Classification [0.0]
  To perform medical image segmentation with deep learning models, it requires training on large image dataset with annotation. To overcome this issue, self-learning annotation scheme was proposed in the two-stage deep learning algorithm. The classification results of the proposed AI framework achieved training accuracy of 93.8% and testing accuracy of 82.42%.
  arXiv  Detail & Related papers  (2020-01-04T09:31:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.