Image-to-Image Translation of Synthetic Samples for Rare Classes

• URL: http://arxiv.org/abs/2106.12212v1
• Date: Wed, 23 Jun 2021 07:57:53 GMT
• Title: Image-to-Image Translation of Synthetic Samples for Rare Classes
• Authors: Edoardo Lanzini and Sara Beery
• Abstract summary: Learning from few examples is a known challenge for deep learning based classification algorithms. One potential approach to increase the training data for these rare classes is to augment the limited real data with synthetic samples. This has been shown to help, but the domain shift between real and synthetic hinders the approaches' efficacy when tested on real data.
• Score: 3.04585143845864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The natural world is long-tailed: rare classes are observed orders of magnitudes less frequently than common ones, leading to highly-imbalanced data where rare classes can have only handfuls of examples. Learning from few examples is a known challenge for deep learning based classification algorithms, and is the focus of the field of low-shot learning. One potential approach to increase the training data for these rare classes is to augment the limited real data with synthetic samples. This has been shown to help, but the domain shift between real and synthetic hinders the approaches' efficacy when tested on real data. We explore the use of image-to-image translation methods to close the domain gap between synthetic and real imagery for animal species classification in data collected from camera traps: motion-activated static cameras used to monitor wildlife. We use low-level feature alignment between source and target domains to make synthetic data for a rare species generated using a graphics engine more "realistic". Compared against a system augmented with unaligned synthetic data, our experiments show a considerable decrease in classification error rates on a rare species.

Related papers

• ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data [0.2302001830524133]
  We use synthetic data generated with a 3D simulator to obtain the first synthetic dataset that can be used for both detection and 2D pose estimation of zebras. We extensively train and benchmark our detection and 2D pose estimation models on multiple real-world and synthetic datasets. These experiments show how the models trained from scratch and only with synthetic data can consistently generalize to real-world images of zebras.
  arXiv  Detail & Related papers  (2024-08-20T13:28:37Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Training Class-Imbalanced Diffusion Model Via Overlap Optimization [55.96820607533968]
  Diffusion models trained on real-world datasets often yield inferior fidelity for tail classes. Deep generative models, including diffusion models, are biased towards classes with abundant training images. We propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
  arXiv  Detail & Related papers  (2024-02-16T16:47:21Z)
• Improving the Effectiveness of Deep Generative Data [5.856292656853396]
  Training a model on purely synthetic images for downstream image processing tasks results in an undesired performance drop compared to training on real data. We propose a new taxonomy to describe factors contributing to this commonly observed phenomenon and investigate it on the popular CIFAR-10 dataset. Our method outperforms baselines on downstream classification tasks both in case of training on synthetic only (Synthetic-to-Real) and training on a mix of real and synthetic data.
  arXiv  Detail & Related papers  (2023-11-07T12:57:58Z)
• Image change detection with only a few samples [7.5780621370948635]
  A major impediment of image change detection task is the lack of large annotated datasets covering a wide variety of scenes. We propose using simple image processing methods for generating synthetic but informative datasets. We then design an early fusion network based on object detection which could outperform the siamese neural network.
  arXiv  Detail & Related papers  (2023-11-07T07:01:35Z)
• ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning [102.46382882098847]
  We first investigate the effects of synthetic data in synthetic-to-real novel view synthesis. We propose to introduce geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints. Our method can render images with higher quality and better fine-grained details, outperforming existing generalizable novel view synthesis methods in terms of PSNR, SSIM, and LPIPS.
  arXiv  Detail & Related papers  (2023-03-20T12:06:14Z)
• Effective Data Augmentation With Diffusion Models [65.09758931804478]
  We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.
  arXiv  Detail & Related papers  (2023-02-07T20:42:28Z)
• Synthetic Data for Object Classification in Industrial Applications [53.180678723280145]
  In object classification, capturing a large number of images per object and in different conditions is not always possible. This work explores the creation of artificial images using a game engine to cope with limited data in the training dataset.
  arXiv  Detail & Related papers  (2022-12-09T11:43:04Z)
• Is synthetic data from generative models ready for image recognition? [69.42645602062024]
  We study whether and how synthetic images generated from state-of-the-art text-to-image generation models can be used for image recognition tasks. We showcase the powerfulness and shortcomings of synthetic data from existing generative models, and propose strategies for better applying synthetic data for recognition tasks.
  arXiv  Detail & Related papers  (2022-10-14T06:54:24Z)
• Synthetic Data for Model Selection [2.4499092754102874]
  We show that synthetic data can be beneficial for model selection. We introduce a novel method to calibrate the synthetic error estimation to fit that of the real domain.
  arXiv  Detail & Related papers  (2021-05-03T09:52:03Z)
• Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation [16.153683223016973]
  We develop an attention module that learns to identify and remove difficult out-of-domain regions in real images. Visualizing the removed regions provides interpretable insights into the synthetic-real domain gap.
  arXiv  Detail & Related papers  (2020-02-27T14:28:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.