Phenotype-preserving metric design for high-content image reconstruction by generative inpainting

• URL: http://arxiv.org/abs/2307.14436v3
• Date: Tue, 22 Aug 2023 17:31:49 GMT
• Title: Phenotype-preserving metric design for high-content image reconstruction by generative inpainting
• Authors: Vaibhav Sharma, Artur Yakimovich
• Abstract summary: We evaluate the state-of-the-art inpainting methods for image restoration in a high-content fluorescence microscopy dataset of cultured cells. We show that architectures like DeepFill V2 and Edge Connect can faithfully restore microscopy images upon fine-tuning with relatively little data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the past decades, automated high-content microscopy demonstrated its ability to deliver large quantities of image-based data powering the versatility of phenotypic drug screening and systems biology applications. However, as the sizes of image-based datasets grew, it became infeasible for humans to control, avoid and overcome the presence of imaging and sample preparation artefacts in the images. While novel techniques like machine learning and deep learning may address these shortcomings through generative image inpainting, when applied to sensitive research data this may come at the cost of undesired image manipulation. Undesired manipulation may be caused by phenomena such as neural hallucinations, to which some artificial neural networks are prone. To address this, here we evaluate the state-of-the-art inpainting methods for image restoration in a high-content fluorescence microscopy dataset of cultured cells with labelled nuclei. We show that architectures like DeepFill V2 and Edge Connect can faithfully restore microscopy images upon fine-tuning with relatively little data. Our results demonstrate that the area of the region to be restored is of higher importance than shape. Furthermore, to control for the quality of restoration, we propose a novel phenotype-preserving metric design strategy. In this strategy, the size and count of the restored biological phenotypes like cell nuclei are quantified to penalise undesirable manipulation. We argue that the design principles of our approach may also generalise to other applications.

Related papers

• Disentangling representations of retinal images with generative models [12.547633373232026]
  We introduce a novel population model for retinal fundus images that disentangles patient attributes from camera effects. Our results show that our model provides a new perspective on the complex relationship between patient attributes and technical confounders in retinal fundus image generation.
  arXiv  Detail & Related papers  (2024-02-29T14:11:08Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• DeepContrast: Deep Tissue Contrast Enhancement using Synthetic Data Degradations and OOD Model Predictions [6.550912532749276]
  We propose a new method to counteract blurring and contrast loss in microscopy images. We first synthetically degraded the quality of microscopy images even further by using an approximate forward model for deep tissue image degradations. We trained a neural network that learned the inverse of this degradation function from our generated pairs of raw and degraded images.
  arXiv  Detail & Related papers  (2023-08-16T13:40:01Z)
• LUCYD: A Feature-Driven Richardson-Lucy Deconvolution Network [0.31402652384742363]
  This paper proposes LUCYD, a novel method for the restoration of volumetric microscopy images. Lucyd combines the Richardson-Lucy deconvolution formula and the fusion of deep features obtained by a fully convolutional network. Our experiments indicate that LUCYD can significantly improve resolution, contrast, and overall quality of microscopy images.
  arXiv  Detail & Related papers  (2023-07-16T10:34:23Z)
• Learning multi-scale functional representations of proteins from single-cell microscopy data [77.34726150561087]
  We show that simple convolutional networks trained on localization classification can learn protein representations that encapsulate diverse functional information. We also propose a robust evaluation strategy to assess quality of protein representations across different scales of biological function.
  arXiv  Detail & Related papers  (2022-05-24T00:00:07Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Sharp-GAN: Sharpness Loss Regularized GAN for Histopathology Image Synthesis [65.47507533905188]
  Conditional generative adversarial networks have been applied to generate synthetic histopathology images. We propose a sharpness loss regularized generative adversarial network to synthesize realistic histopathology images.
  arXiv  Detail & Related papers  (2021-10-27T18:54:25Z)
• From augmented microscopy to the topological transformer: a new approach in cell image analysis for Alzheimer's research [0.0]
  Cell image analysis is crucial in Alzheimer's research to detect the presence of A$beta$ protein inhibiting cell function. We first found Unet is most suitable in augmented microscopy by comparing performance in multi-class semantics segmentation. We develop the augmented microscopy method to capture nuclei in a brightfield image and the transformer using Unet model to convert an input image into a sequence of topological information.
  arXiv  Detail & Related papers  (2021-08-03T16:59:33Z)
• Deep learning-based super-resolution fluorescence microscopy on small datasets [20.349746411933495]
  Deep learning has shown the potentials to reduce the technical barrier and obtain super-resolution from diffraction-limited images. We demonstrate a new convolutional neural network-based approach that is successfully trained with small datasets and super-resolution images. This model can be applied to other biomedical imaging modalities such as MRI and X-ray imaging, where obtaining large training datasets is challenging.
  arXiv  Detail & Related papers  (2021-03-07T03:17:47Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Neural Sparse Representation for Image Restoration [116.72107034624344]
  Inspired by the robustness and efficiency of sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks. Our method structurally enforces sparsity constraints upon hidden neurons. Experiments show that sparse representation is crucial in deep neural networks for multiple image restoration tasks.
  arXiv  Detail & Related papers  (2020-06-08T05:15:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.