Informative Sample Mining Network for Multi-Domain Image-to-Image Translation

• URL: http://arxiv.org/abs/2001.01173v4
• Date: Sun, 20 Sep 2020 09:56:53 GMT
• Title: Informative Sample Mining Network for Multi-Domain Image-to-Image Translation
• Authors: Jie Cao, Huaibo Huang, Yi Li, Ran He, Zhenan Sun
• Abstract summary: We show that improving the sample selection strategy is an effective solution for image-to-image translation tasks. We propose a novel multi-stage sample training scheme to reduce sample hardness while preserving sample informativeness.
• Score: 101.01649070998532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The performance of multi-domain image-to-image translation has been significantly improved by recent progress in deep generative models. Existing approaches can use a unified model to achieve translations between all the visual domains. However, their outcomes are far from satisfying when there are large domain variations. In this paper, we reveal that improving the sample selection strategy is an effective solution. To select informative samples, we dynamically estimate sample importance during the training of Generative Adversarial Networks, presenting Informative Sample Mining Network. We theoretically analyze the relationship between the sample importance and the prediction of the global optimal discriminator. Then a practical importance estimation function for general conditions is derived. Furthermore, we propose a novel multi-stage sample training scheme to reduce sample hardness while preserving sample informativeness. Extensive experiments on a wide range of specific image-to-image translation tasks are conducted, and the results demonstrate our superiority over current state-of-the-art methods.

Related papers

• Integrated Image-Text Based on Semi-supervised Learning for Small Sample Instance Segmentation [1.3157419797035321]
  The article proposes a novel small sample instance segmentation solution from the perspective of maximizing the utilization of existing information. First, it helps the model fully utilize unlabeled data by learning to generate pseudo labels, increasing the number of available samples. Second, by integrating the features of text and image, more accurate classification results can be obtained.
  arXiv  Detail & Related papers  (2024-10-21T14:44:08Z)
• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization [65.57380193070574]
  Vision-language pre-training models are vulnerable to multi-modal adversarial examples. Recent works have indicated that leveraging data augmentation and image-text modal interactions can enhance the transferability of adversarial examples. We propose an Optimal Transport-based Adversarial Attack, dubbed OT-Attack.
  arXiv  Detail & Related papers  (2023-12-07T16:16:50Z)
• Improving Diversity in Zero-Shot GAN Adaptation with Semantic Variations [61.132408427908175]
  zero-shot GAN adaptation aims to reuse well-trained generators to synthesize images of an unseen target domain. With only a single representative text feature instead of real images, the synthesized images gradually lose diversity. We propose a novel method to find semantic variations of the target text in the CLIP space.
  arXiv  Detail & Related papers  (2023-08-21T08:12:28Z)
• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• Multimodal Data Augmentation for Image Captioning using Diffusion Models [12.221685807426264]
  We propose a data augmentation method, leveraging a text-to-image model called Stable Diffusion, to expand the training set. Experiments on the MS COCO dataset demonstrate the advantages of our approach over several benchmark methods. Further improvement regarding the training efficiency and effectiveness can be obtained after intentionally filtering the generated data.
  arXiv  Detail & Related papers  (2023-05-03T01:57:33Z)
• Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets [1.07539359851877]
  In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches.
  arXiv  Detail & Related papers  (2023-01-02T14:17:08Z)
• Conditional Variational Autoencoder for Learned Image Reconstruction [5.487951901731039]
  We develop a novel framework that approximates the posterior distribution of the unknown image at each query observation. It handles implicit noise models and priors, it incorporates the data formation process (i.e., the forward operator), and the learned reconstructive properties are transferable between different datasets.
  arXiv  Detail & Related papers  (2021-10-22T10:02:48Z)
• An Unsupervised Sampling Approach for Image-Sentence Matching Using Document-Level Structural Information [64.66785523187845]
  We focus on the problem of unsupervised image-sentence matching. Existing research explores to utilize document-level structural information to sample positive and negative instances for model training. We propose a new sampling strategy to select additional intra-document image-sentence pairs as positive or negative samples.
  arXiv  Detail & Related papers  (2021-03-21T05:43:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.