Anatomical Invariance Modeling and Semantic Alignment for
Self-supervised Learning in 3D Medical Image Analysis
- URL: http://arxiv.org/abs/2302.05615v3
- Date: Thu, 17 Aug 2023 16:18:39 GMT
- Title: Anatomical Invariance Modeling and Semantic Alignment for
Self-supervised Learning in 3D Medical Image Analysis
- Authors: Yankai Jiang, Mingze Sun, Heng Guo, Xiaoyu Bai, Ke Yan, Le Lu and
Minfeng Xu
- Abstract summary: Self-supervised learning (SSL) has recently achieved promising performance for 3D medical image analysis tasks.
Most current methods follow existing SSL paradigm originally designed for photographic or natural images.
We propose a new self-supervised learning framework, namely Alice, that explicitly fulfills Anatomical invariance modeling and semantic alignment.
- Score: 6.87667643104543
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Self-supervised learning (SSL) has recently achieved promising performance
for 3D medical image analysis tasks. Most current methods follow existing SSL
paradigm originally designed for photographic or natural images, which cannot
explicitly and thoroughly exploit the intrinsic similar anatomical structures
across varying medical images. This may in fact degrade the quality of learned
deep representations by maximizing the similarity among features containing
spatial misalignment information and different anatomical semantics. In this
work, we propose a new self-supervised learning framework, namely Alice, that
explicitly fulfills Anatomical invariance modeling and semantic alignment via
elaborately combining discriminative and generative objectives. Alice
introduces a new contrastive learning strategy which encourages the similarity
between views that are diversely mined but with consistent high-level
semantics, in order to learn invariant anatomical features. Moreover, we design
a conditional anatomical feature alignment module to complement corrupted
embeddings with globally matched semantics and inter-patch topology
information, conditioned by the distribution of local image content, which
permits to create better contrastive pairs. Our extensive quantitative
experiments on three 3D medical image analysis tasks demonstrate and validate
the performance superiority of Alice, surpassing the previous best SSL
counterpart methods and showing promising ability for united representation
learning. Codes are available at https://github.com/alibaba-damo-academy/alice.
Related papers
- OPTiML: Dense Semantic Invariance Using Optimal Transport for Self-Supervised Medical Image Representation [6.4136876268620115]
Self-supervised learning (SSL) has emerged as a promising technique for medical image analysis due to its ability to learn without annotations.
We introduce a novel SSL framework OPTiML, employing optimal transport (OT), to capture the dense semantic invariance and fine-grained details.
Our empirical results reveal OPTiML's superiority over state-of-the-art methods across all evaluated tasks.
arXiv Detail & Related papers (2024-04-18T02:59:48Z) - Overcoming Dimensional Collapse in Self-supervised Contrastive Learning
for Medical Image Segmentation [2.6764957223405657]
We investigate the application of contrastive learning to the domain of medical image analysis.
Our findings reveal that MoCo v2, a state-of-the-art contrastive learning method, encounters dimensional collapse when applied to medical images.
To address this, we propose two key contributions: local feature learning and feature decorrelation.
arXiv Detail & Related papers (2024-02-22T15:02:13Z) - Learning Anatomically Consistent Embedding for Chest Radiography [4.990778682575127]
This paper introduces a novel SSL approach, called PEAC (patch embedding of anatomical consistency), for medical image analysis.
Specifically, we propose to learn global and local consistencies via stable grid-based matching, transfer pre-trained PEAC models to diverse downstream tasks.
We extensively demonstrate that PEAC achieves significantly better performance than the existing state-of-the-art fully/self-supervised methods.
arXiv Detail & Related papers (2023-12-01T04:07:12Z) - Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic
Image Synthesis [139.2216271759332]
We propose a novel ECGAN for the challenging semantic image synthesis task.
The semantic labels do not provide detailed structural information, making it challenging to synthesize local details and structures.
The widely adopted CNN operations such as convolution, down-sampling, and normalization usually cause spatial resolution loss.
We propose a novel contrastive learning method, which aims to enforce pixel embeddings belonging to the same semantic class to generate more similar image content.
arXiv Detail & Related papers (2023-07-22T14:17:19Z) - GraVIS: Grouping Augmented Views from Independent Sources for
Dermatology Analysis [52.04899592688968]
We propose GraVIS, which is specifically optimized for learning self-supervised features from dermatology images.
GraVIS significantly outperforms its transfer learning and self-supervised learning counterparts in both lesion segmentation and disease classification tasks.
arXiv Detail & Related papers (2023-01-11T11:38:37Z) - PCRLv2: A Unified Visual Information Preservation Framework for
Self-supervised Pre-training in Medical Image Analysis [56.63327669853693]
We propose to incorporate the task of pixel restoration for explicitly encoding more pixel-level information into high-level semantics.
We also address the preservation of scale information, a powerful tool in aiding image understanding.
The proposed unified SSL framework surpasses its self-supervised counterparts on various tasks.
arXiv Detail & Related papers (2023-01-02T17:47:27Z) - Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks.
In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics.
Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
arXiv Detail & Related papers (2022-09-19T09:43:19Z) - Lesion-Aware Contrastive Representation Learning for Histopathology
Whole Slide Images Analysis [16.264758789726223]
We propose a novel contrastive representation learning framework named Lesion-Aware Contrastive Learning (LACL) for histopathology whole slide image analysis.
The experimental results demonstrate that LACL achieves the best performance in histopathology image representation learning on different datasets.
arXiv Detail & Related papers (2022-06-27T08:39:51Z) - Unsupervised Domain Adaptation with Contrastive Learning for OCT
Segmentation [49.59567529191423]
We propose a novel semi-supervised learning framework for segmentation of volumetric images from new unlabeled domains.
We jointly use supervised and contrastive learning, also introducing a contrastive pairing scheme that leverages similarity between nearby slices in 3D.
arXiv Detail & Related papers (2022-03-07T19:02:26Z) - Learning Semantics-enriched Representation via Self-discovery,
Self-classification, and Self-restoration [12.609383051645887]
We train deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images.
We examine our Semantic Genesis with all the publicly-available pre-trained models, by either self-supervision or fully supervision, on the six distinct target tasks.
Our experiments demonstrate that Semantic Genesis significantly exceeds all of its 3D counterparts as well as the de facto ImageNet-based transfer learning in 2D.
arXiv Detail & Related papers (2020-07-14T10:36:10Z) - Pathological Retinal Region Segmentation From OCT Images Using Geometric
Relation Based Augmentation [84.7571086566595]
We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape.
The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
arXiv Detail & Related papers (2020-03-31T11:50:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.