Synthetic dual image generation for reduction of labeling efforts in semantic segmentation of micrographs with a customized metric function

• URL: http://arxiv.org/abs/2408.00707v1
• Date: Thu, 1 Aug 2024 16:54:11 GMT
• Title: Synthetic dual image generation for reduction of labeling efforts in semantic segmentation of micrographs with a customized metric function
• Authors: Matias Oscar Volman Stern, Dominic Hohs, Andreas Jansche, Timo Bernthaler, Gerhard Schneider,
• Abstract summary: Training semantic segmentation models for material analysis requires micrographs and their corresponding masks. We demonstrate a workflow for the improvement of semantic segmentation models through the generation of synthetic microstructural images in conjunction with masks. The approach could be generalized to various types of image data such as it serves as a user-friendly solution for training models with a small number of real images.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training of semantic segmentation models for material analysis requires micrographs and their corresponding masks. It is quite unlikely that perfect masks will be drawn, especially at the edges of objects, and sometimes the amount of data that can be obtained is small, since only a few samples are available. These aspects make it very problematic to train a robust model. We demonstrate a workflow for the improvement of semantic segmentation models of micrographs through the generation of synthetic microstructural images in conjunction with masks. The workflow only requires joining a few micrographs with their respective masks to create the input for a Vector Quantised-Variational AutoEncoder model that includes an embedding space, which is trained such that a generative model (PixelCNN) learns the distribution of each input, transformed into discrete codes, and can be used to sample new codes. The latter will eventually be decoded by VQ-VAE to generate images alongside corresponding masks for semantic segmentation. To evaluate the synthetic data, we have trained U-Net models with different amounts of these synthetic data in conjunction with real data. These models were then evaluated using non-synthetic images only. Additionally, we introduce a customized metric derived from the mean Intersection over Union (mIoU). The proposed metric prevents a few falsely predicted pixels from greatly reducing the value of the mIoU. We have achieved a reduction in sample preparation and acquisition times, as well as the efforts, needed for image processing and labeling tasks, are less when it comes to training semantic segmentation model. The approach could be generalized to various types of image data such that it serves as a user-friendly solution for training models with a small number of real images.

Related papers

• Sequential Modeling Enables Scalable Learning for Large Vision Models [120.91839619284431]
  We introduce a novel sequential modeling approach which enables learning a Large Vision Model (LVM) without making use of any linguistic data. We define a common format, "visual sentences", in which we can represent raw images and videos as well as annotated data sources.
  arXiv  Detail & Related papers  (2023-12-01T18:59:57Z)
• With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning [47.96387857237473]
  We devise a network which can perform attention over activations obtained while processing other training samples. Our memory models the distribution of past keys and values through the definition of prototype vectors. We demonstrate that our proposal can increase the performance of an encoder-decoder Transformer by 3.7 CIDEr points both when training in cross-entropy only and when fine-tuning with self-critical sequence training.
  arXiv  Detail & Related papers  (2023-08-23T18:53:00Z)
• Microscopy Image Segmentation via Point and Shape Regularized Data Synthesis [9.47802391546853]
  We develop a unified pipeline for microscopy image segmentation using synthetically generated training data. Our framework achieves comparable results to models trained on authentic microscopy images with dense labels.
  arXiv  Detail & Related papers  (2023-08-18T22:00:53Z)
• DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models [61.906934570771256]
  We present a generic dataset generation model that can produce diverse synthetic images and perception annotations. Our method builds upon the pre-trained diffusion model and extends text-guided image synthesis to perception data generation. We show that the rich latent code of the diffusion model can be effectively decoded as accurate perception annotations using a decoder module.
  arXiv  Detail & Related papers  (2023-08-11T14:38:11Z)
• One-Shot Synthesis of Images and Segmentation Masks [28.119303696418882]
  Joint synthesis of images and segmentation masks with generative adversarial networks (GANs) is promising to reduce the effort needed for collecting image data with pixel-wise annotations. To learn high-fidelity image-mask synthesis, existing GAN approaches first need a pre-training phase requiring large amounts of image data. We introduce our OSMIS model which enables the synthesis of segmentation masks that are precisely aligned to the generated images in the one-shot regime.
  arXiv  Detail & Related papers  (2022-09-15T18:00:55Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)
• Descriptive Modeling of Textiles using FE Simulations and Deep Learning [0.0]
  We propose a novel and fully automated method for extracting the yarn geometrical features in woven composites. The proposed approach employs two deep neural network architectures (U-Net and Mask RCNN) Experimental results show that our method is accurate and robust for performing yarn instance segmentation on CT images.
  arXiv  Detail & Related papers  (2021-06-26T09:32:24Z)
• Autoregressive Unsupervised Image Segmentation [8.894935073145252]
  We propose a new unsupervised image segmentation approach based on mutual information between different views constructed of the inputs. The proposed method outperforms current state-of-the-art on unsupervised image segmentation.
  arXiv  Detail & Related papers  (2020-07-16T10:47:40Z)
• Set Based Stochastic Subsampling [85.5331107565578]
  We propose a set-based two-stage end-to-end neural subsampling model that is jointly optimized with an textitarbitrary downstream task network. We show that it outperforms the relevant baselines under low subsampling rates on a variety of tasks including image classification, image reconstruction, function reconstruction and few-shot classification.
  arXiv  Detail & Related papers  (2020-06-25T07:36:47Z)
• Locally Masked Convolution for Autoregressive Models [107.4635841204146]
  LMConv is a simple modification to the standard 2D convolution that allows arbitrary masks to be applied to the weights at each location in the image. We learn an ensemble of distribution estimators that share parameters but differ in generation order, achieving improved performance on whole-image density estimation.
  arXiv  Detail & Related papers  (2020-06-22T17:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.