Convolution-Free Medical Image Segmentation using Transformers
- URL: http://arxiv.org/abs/2102.13645v1
- Date: Fri, 26 Feb 2021 18:49:13 GMT
- Title: Convolution-Free Medical Image Segmentation using Transformers
- Authors: Davood Karimi, Serge Vasylechko, Ali Gholipour
- Abstract summary: We show that a different method, based entirely on self-attention between neighboring image patches, can achieve competitive or better results.
We show that the proposed model can achieve segmentation accuracies that are better than the state of the art CNNs on three datasets.
- Score: 8.130670465411239
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Like other applications in computer vision, medical image segmentation has
been most successfully addressed using deep learning models that rely on the
convolution operation as their main building block. Convolutions enjoy
important properties such as sparse interactions, weight sharing, and
translation equivariance. These properties give convolutional neural networks
(CNNs) a strong and useful inductive bias for vision tasks. In this work we
show that a different method, based entirely on self-attention between
neighboring image patches and without any convolution operations, can achieve
competitive or better results. Given a 3D image block, our network divides it
into $n^3$ 3D patches, where $n=3 \text{ or } 5$ and computes a 1D embedding
for each patch. The network predicts the segmentation map for the center patch
of the block based on the self-attention between these patch embeddings. We
show that the proposed model can achieve segmentation accuracies that are
better than the state of the art CNNs on three datasets. We also propose
methods for pre-training this model on large corpora of unlabeled images. Our
experiments show that with pre-training the advantage of our proposed network
over CNNs can be significant when labeled training data is small.
Related papers
- Bayesian Self-Training for Semi-Supervised 3D Segmentation [59.544558398992386]
3D segmentation is a core problem in computer vision.
densely labeling 3D point clouds to employ fully-supervised training remains too labor intensive and expensive.
Semi-supervised training provides a more practical alternative, where only a small set of labeled data is given, accompanied by a larger unlabeled set.
arXiv Detail & Related papers (2024-09-12T14:54:31Z) - Contextual Embedding Learning to Enhance 2D Networks for Volumetric Image Segmentation [5.995633685952995]
2D convolutional neural networks (CNNs) can hardly exploit the spatial correlation of volumetric data.
We propose a contextual embedding learning approach to facilitate 2D CNNs capturing spatial information properly.
Our approach leverages the learned embedding and the slice-wisely neighboring matching as a soft cue to guide the network.
arXiv Detail & Related papers (2024-04-02T08:17:39Z) - Two-Stream Graph Convolutional Network for Intra-oral Scanner Image
Segmentation [133.02190910009384]
We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes.
Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
arXiv Detail & Related papers (2022-04-19T10:41:09Z) - Deep ensembles in bioimage segmentation [74.01883650587321]
In this work, we propose an ensemble of convolutional neural networks (CNNs)
In ensemble methods, many different models are trained and then used for classification, the ensemble aggregates the outputs of the single classifiers.
The proposed ensemble is implemented by combining different backbone networks using the DeepLabV3+ and HarDNet environment.
arXiv Detail & Related papers (2021-12-24T05:54:21Z) - Patch-based medical image segmentation using Quantum Tensor Networks [1.5899411215927988]
We formulate image segmentation in a supervised setting with tensor networks.
The key idea is to first lift the pixels in image patches to exponentially high dimensional feature spaces.
The performance of the proposed model is evaluated on three 2D- and one 3D- biomedical imaging datasets.
arXiv Detail & Related papers (2021-09-15T07:54:05Z) - Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI
Classification [1.714108629548376]
We propose to leverage continuous proxy metadata, in the contrastive learning framework, by introducing a new loss called y-Aware InfoNCE loss.
A 3D CNN model pre-trained on $104$ multi-site healthy brain MRI scans can extract relevant features for three classification tasks.
When fine-tuned, it also outperforms 3D CNN trained from scratch on these tasks, as well as state-of-the-art self-supervised methods.
arXiv Detail & Related papers (2021-06-16T14:17:04Z) - CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image
Segmentation [95.51455777713092]
Convolutional neural networks (CNNs) have been the de facto standard for nowadays 3D medical image segmentation.
We propose a novel framework that efficiently bridges a bf Convolutional neural network and a bf Transformer bf (CoTr) for accurate 3D medical image segmentation.
arXiv Detail & Related papers (2021-03-04T13:34:22Z) - Learning Hybrid Representations for Automatic 3D Vessel Centerline
Extraction [57.74609918453932]
Automatic blood vessel extraction from 3D medical images is crucial for vascular disease diagnoses.
Existing methods may suffer from discontinuities of extracted vessels when segmenting such thin tubular structures from 3D images.
We argue that preserving the continuity of extracted vessels requires to take into account the global geometry.
We propose a hybrid representation learning approach to address this challenge.
arXiv Detail & Related papers (2020-12-14T05:22:49Z) - Test-Time Adaptable Neural Networks for Robust Medical Image
Segmentation [9.372152932156293]
Convolutional Neural Networks (CNNs) work very well for supervised learning problems.
In medical image segmentation, this premise is violated when there is a mismatch between training and test images in terms of their acquisition details.
We design the segmentation CNN as a concatenation of two sub-networks: a relatively shallow image normalization CNN, followed by a deep CNN that segments the normalized image.
arXiv Detail & Related papers (2020-04-09T16:57:27Z) - 3D medical image segmentation with labeled and unlabeled data using
autoencoders at the example of liver segmentation in CT images [58.720142291102135]
This work investigates the potential of autoencoder-extracted features to improve segmentation with a convolutional neural network.
A convolutional autoencoder was used to extract features from unlabeled data and a multi-scale, fully convolutional CNN was used to perform the target task of 3D liver segmentation in CT images.
arXiv Detail & Related papers (2020-03-17T20:20:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.