Learning Generative Models of Tissue Organization with Supervised GANs
- URL: http://arxiv.org/abs/2004.00140v1
- Date: Tue, 31 Mar 2020 22:22:58 GMT
- Title: Learning Generative Models of Tissue Organization with Supervised GANs
- Authors: Ligong Han, Robert F. Murphy, and Deva Ramanan
- Abstract summary: A key step in understanding the spatial organization of cells and tissues is the ability to construct generative models that accurately reflect that organization.
In this paper, we focus on building generative models of electron microscope (EM) images in which the positions of cell membranes and mitochondria have been densely annotated.
We propose a two-stage procedure that produces realistic images using Generative Adversarial Networks (or GANs) in a supervised way.
- Score: 46.569795520982325
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: A key step in understanding the spatial organization of cells and tissues is
the ability to construct generative models that accurately reflect that
organization. In this paper, we focus on building generative models of electron
microscope (EM) images in which the positions of cell membranes and
mitochondria have been densely annotated, and propose a two-stage procedure
that produces realistic images using Generative Adversarial Networks (or GANs)
in a supervised way. In the first stage, we synthesize a label "image" given a
noise "image" as input, which then provides supervision for EM image synthesis
in the second stage. The full model naturally generates label-image pairs. We
show that accurate synthetic EM images are produced using assessment via (1)
shape features and global statistics, (2) segmentation accuracies, and (3) user
studies. We also demonstrate further improvements by enforcing a reconstruction
loss on intermediate synthetic labels and thus unifying the two stages into one
single end-to-end framework.
Related papers
- Forgery-aware Adaptive Transformer for Generalizable Synthetic Image
Detection [106.39544368711427]
We study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods.
We present a novel forgery-aware adaptive transformer approach, namely FatFormer.
Our approach tuned on 4-class ProGAN data attains an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
arXiv Detail & Related papers (2023-12-27T17:36:32Z) - Unlocking Pre-trained Image Backbones for Semantic Image Synthesis [29.688029979801577]
We propose a new class of GAN discriminators for semantic image synthesis that generates highly realistic images.
Our model, which we dub DP-SIMS, achieves state-of-the-art results in terms of image quality and consistency with the input label maps on ADE-20K, COCO-Stuff, and Cityscapes.
arXiv Detail & Related papers (2023-12-20T09:39:19Z) - Bi-Modality Medical Image Synthesis Using Semi-Supervised Sequential
Generative Adversarial Networks [35.358653509217994]
We propose a bi-modality medical image synthesis approach based on sequential generative adversarial network (GAN) and semi-supervised learning.
Our approach consists of two generative modules that synthesize images of the two modalities in a sequential order.
Visual and quantitative results demonstrate the superiority of our method to the state-of-the-art methods.
arXiv Detail & Related papers (2023-08-27T10:39:33Z) - Pathology Synthesis of 3D-Consistent Cardiac MR Images using 2D VAEs and
GANs [0.5039813366558306]
We propose a method for generating labeled data for the application of supervised deep-learning (DL) training.
The image synthesis consists of label deformation and label-to-image translation tasks.
We demonstrate that such an approach could provide a solution to diversify and enrich an available database of cardiac MR images.
arXiv Detail & Related papers (2022-09-09T10:17:49Z) - Semi-Supervised Segmentation of Mitochondria from Electron Microscopy
Images Using Spatial Continuity [3.631638087834872]
We propose a semi-supervised deep learning model that segments mitochondria by leveraging the spatial continuity of their structural, morphological, and contextual information.
Our model achieves performance similar to that of state-of-the-art fully supervised models but requires only 20% of their annotated training data.
arXiv Detail & Related papers (2022-06-06T06:52:19Z) - A Deep Learning Generative Model Approach for Image Synthesis of Plant
Leaves [62.997667081978825]
We generate via advanced Deep Learning (DL) techniques artificial leaf images in an automatized way.
We aim to dispose of a source of training samples for AI applications for modern crop management.
arXiv Detail & Related papers (2021-11-05T10:53:35Z) - IMAGINE: Image Synthesis by Image-Guided Model Inversion [79.4691654458141]
We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images.
We leverage the knowledge of image semantics from a pre-trained classifier to achieve plausible generations.
IMAGINE enables the synthesis procedure to simultaneously 1) enforce semantic specificity constraints during the synthesis, 2) produce realistic images without generator training, and 3) give users intuitive control over the generation process.
arXiv Detail & Related papers (2021-04-13T02:00:24Z) - Semantic Segmentation with Generative Models: Semi-Supervised Learning
and Strong Out-of-Domain Generalization [112.68171734288237]
We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels.
We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images.
We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
arXiv Detail & Related papers (2021-04-12T21:41:25Z) - You Only Need Adversarial Supervision for Semantic Image Synthesis [84.83711654797342]
We propose a novel, simplified GAN model, which needs only adversarial supervision to achieve high quality results.
We show that images synthesized by our model are more diverse and follow the color and texture of real images more closely.
arXiv Detail & Related papers (2020-12-08T23:00:48Z) - Improving Augmentation and Evaluation Schemes for Semantic Image
Synthesis [16.097324852253912]
We introduce a novel augmentation scheme designed specifically for generative adversarial networks (GANs)
We propose to randomly warp object shapes in the semantic label maps used as an input to the generator.
The local shape discrepancies between the warped and non-warped label maps and images enable the GAN to learn better the structural and geometric details of the scene.
arXiv Detail & Related papers (2020-11-25T10:55:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.