Source Identification: A Self-Supervision Task for Dense Prediction
- URL: http://arxiv.org/abs/2307.02238v1
- Date: Wed, 5 Jul 2023 12:27:58 GMT
- Title: Source Identification: A Self-Supervision Task for Dense Prediction
- Authors: Shuai Chen and Subhradeep Kayal and Marleen de Bruijne
- Abstract summary: We propose a new self-supervision task called source identification (SI)
Synthetic images are generated by fusing multiple source images and the network's task is to reconstruct the original images, given the fused images.
We validate our method on two medical image segmentation tasks: brain tumor segmentation and white matter hyperintensities segmentation.
- Score: 8.744460886823322
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The paradigm of self-supervision focuses on representation learning from raw
data without the need of labor-consuming annotations, which is the main
bottleneck of current data-driven methods. Self-supervision tasks are often
used to pre-train a neural network with a large amount of unlabeled data and
extract generic features of the dataset. The learned model is likely to contain
useful information which can be transferred to the downstream main task and
improve performance compared to random parameter initialization. In this paper,
we propose a new self-supervision task called source identification (SI), which
is inspired by the classic blind source separation problem. Synthetic images
are generated by fusing multiple source images and the network's task is to
reconstruct the original images, given the fused images. A proper understanding
of the image content is required to successfully solve the task. We validate
our method on two medical image segmentation tasks: brain tumor segmentation
and white matter hyperintensities segmentation. The results show that the
proposed SI task outperforms traditional self-supervision tasks for dense
predictions including inpainting, pixel shuffling, intensity shift, and
super-resolution. Among variations of the SI task fusing images of different
types, fusing images from different patients performs best.
Related papers
- Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting [49.87694319431288]
Generalist segmentation models are increasingly favored for diverse tasks involving various objects from different image sources.
We propose a Comprehensive Generative (CGR) framework that restores appearance and semantic knowledge by synthesizing image-mask pairs.
Experiments on incremental tasks (cardiac, fundus and prostate segmentation) show its clear advantage for alleviating concurrent appearance and semantic forgetting.
arXiv Detail & Related papers (2024-06-28T10:05:58Z) - Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement
Learning [53.00683059396803]
Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images.
We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy.
Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
arXiv Detail & Related papers (2023-10-06T10:40:46Z) - GenSelfDiff-HIS: Generative Self-Supervision Using Diffusion for Histopathological Image Segmentation [5.049466204159458]
Self-supervised learning (SSL) is an alternative paradigm that provides some respite by constructing models utilizing only the unannotated data.
In this paper, we propose an SSL approach for segmenting histopathological images via generative diffusion models.
Our method is based on the observation that diffusion models effectively solve an image-to-image translation task akin to a segmentation task.
arXiv Detail & Related papers (2023-09-04T09:49:24Z) - Learning to Exploit Temporal Structure for Biomedical Vision-Language
Processing [53.89917396428747]
Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities.
We explicitly account for prior images and reports when available during both training and fine-tuning.
Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
arXiv Detail & Related papers (2023-01-11T16:35:33Z) - Two-Stream Graph Convolutional Network for Intra-oral Scanner Image
Segmentation [133.02190910009384]
We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes.
Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
arXiv Detail & Related papers (2022-04-19T10:41:09Z) - Semantic-Aware Generation for Self-Supervised Visual Representation
Learning [116.5814634936371]
We advocate for Semantic-aware Generation (SaGe) to facilitate richer semantics rather than details to be preserved in the generated image.
SaGe complements the target network with view-specific features and thus alleviates the semantic degradation brought by intensive data augmentations.
We execute SaGe on ImageNet-1K and evaluate the pre-trained models on five downstream tasks including nearest neighbor test, linear classification, and fine-scaled image recognition.
arXiv Detail & Related papers (2021-11-25T16:46:13Z) - Self-Supervised Generative Style Transfer for One-Shot Medical Image
Segmentation [10.634870214944055]
In medical image segmentation, supervised deep networks' success comes at the cost of requiring abundant labeled data.
We propose a novel volumetric self-supervised learning for data augmentation capable of synthesizing volumetric image-segmentation pairs.
Our work's central tenet benefits from a combined view of one-shot generative learning and the proposed self-supervised training strategy.
arXiv Detail & Related papers (2021-10-05T15:28:42Z) - Enhancing MR Image Segmentation with Realistic Adversarial Data
Augmentation [17.539828821476224]
We propose an adversarial data augmentation approach to improve the efficiency in utilizing training data.
We present a generic task-driven learning framework, which jointly optimize a data augmentation model and a segmentation network during training.
The proposed adversarial data augmentation does not rely on generative networks and can be used as a plug-in module in general segmentation networks.
arXiv Detail & Related papers (2021-08-07T11:32:37Z) - Hierarchical Self-Supervised Learning for Medical Image Segmentation
Based on Multi-Domain Data Aggregation [23.616336382437275]
We propose Hierarchical Self-Supervised Learning (HSSL) for medical image segmentation.
We first aggregate a dataset from several medical challenges, then pre-train the network in a self-supervised manner, and finally fine-tune on labeled data.
Compared to learning from scratch, our new method yields better performance on various tasks.
arXiv Detail & Related papers (2021-07-10T18:17:57Z) - Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation.
We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths.
In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
arXiv Detail & Related papers (2020-12-09T12:40:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.