Assessing a Single Image in Reference-Guided Image Synthesis

• URL: http://arxiv.org/abs/2112.04163v1
• Date: Wed, 8 Dec 2021 08:22:14 GMT
• Title: Assessing a Single Image in Reference-Guided Image Synthesis
• Authors: Jiayi Guo, Chaoqun Du, Jiangshan Wang, Huijuan Huang, Pengfei Wan, Gao Huang
• Abstract summary: We propose a learning-based framework, Reference-guided Image Synthesis Assessment (RISA) to quantitatively evaluate the quality of a single generated image. As this annotation is too coarse as a supervision signal, we introduce two techniques: 1) a pixel-wise scheme to refine the coarse labels, and 2) multiple binary classifiers to replace a na"ive regressor. RISA is highly consistent with human preference and transfers well across models.
• Score: 14.936460594115953
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Assessing the performance of Generative Adversarial Networks (GANs) has been an important topic due to its practical significance. Although several evaluation metrics have been proposed, they generally assess the quality of the whole generated image distribution. For Reference-guided Image Synthesis (RIS) tasks, i.e., rendering a source image in the style of another reference image, where assessing the quality of a single generated image is crucial, these metrics are not applicable. In this paper, we propose a general learning-based framework, Reference-guided Image Synthesis Assessment (RISA) to quantitatively evaluate the quality of a single generated image. Notably, the training of RISA does not require human annotations. In specific, the training data for RISA are acquired by the intermediate models from the training procedure in RIS, and weakly annotated by the number of models' iterations, based on the positive correlation between image quality and iterations. As this annotation is too coarse as a supervision signal, we introduce two techniques: 1) a pixel-wise interpolation scheme to refine the coarse labels, and 2) multiple binary classifiers to replace a na\"ive regressor. In addition, an unsupervised contrastive loss is introduced to effectively capture the style similarity between a generated image and its reference image. Empirical results on various datasets demonstrate that RISA is highly consistent with human preference and transfers well across models.

Related papers

• Attention Down-Sampling Transformer, Relative Ranking and Self-Consistency for Blind Image Quality Assessment [17.04649536069553]
  No-reference image quality assessment is a challenging domain that addresses estimating image quality without the original reference. We introduce an improved mechanism to extract local and non-local information from images via different transformer encoders and CNNs. A self-consistency approach to self-supervision is presented, explicitly addressing the degradation of no-reference image quality assessment (NR-IQA) models.
  arXiv  Detail & Related papers  (2024-09-11T09:08:43Z)
• CrossScore: Towards Multi-View Image Evaluation and Scoring [24.853612457257697]
  Cross-reference image quality assessment method fills the gap in the image assessment landscape. Our method enables accurate image quality assessment without requiring ground truth references.
  arXiv  Detail & Related papers  (2024-04-22T17:59:36Z)
• Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerability [21.355484227864466]
  We investigate the relationship between the representation space and input space around generated images. We introduce a new metric to evaluating image-generative models called anomaly score (AS)
  arXiv  Detail & Related papers  (2023-12-17T07:33:06Z)
• Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based Image Retrieval [69.46139774646308]
  This paper studies the problem of zero-shot sketch-based image retrieval (ZS-SBIR) It aims to use sketches from unseen categories as queries to match the images of the same category. We propose a novel Symmetrical Bidirectional Knowledge Alignment for zero-shot sketch-based image retrieval (SBKA)
  arXiv  Detail & Related papers  (2023-12-16T04:50:34Z)
• Diversified in-domain synthesis with efficient fine-tuning for few-shot classification [64.86872227580866]
  Few-shot image classification aims to learn an image classifier using only a small set of labeled examples per class. We propose DISEF, a novel approach which addresses the generalization challenge in few-shot learning using synthetic data. We validate our method in ten different benchmarks, consistently outperforming baselines and establishing a new state-of-the-art for few-shot classification.
  arXiv  Detail & Related papers  (2023-12-05T17:18:09Z)
• ARNIQA: Learning Distortion Manifold for Image Quality Assessment [28.773037051085318]
  No-Reference Image Quality Assessment (NR-IQA) aims to develop methods to measure image quality in alignment with human perception without the need for a high-quality reference image. We propose a self-supervised approach named ARNIQA for modeling the image distortion manifold to obtain quality representations in an intrinsic manner.
  arXiv  Detail & Related papers  (2023-10-20T17:22:25Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)
• Compound Frechet Inception Distance for Quality Assessment of GAN Created Images [7.628527132779575]
  One notable application of GANs is developing fake human faces, also known as "deep fakes" Measuring the quality of the generated images is inherently subjective but attempts to objectify quality using standardized metrics have been made. We propose to improve the robustness of the evaluation process by integrating lower-level features to cover a wider array of visual defects.
  arXiv  Detail & Related papers  (2021-06-16T06:53:27Z)
• IMAGINE: Image Synthesis by Image-Guided Model Inversion [79.4691654458141]
  We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images. We leverage the knowledge of image semantics from a pre-trained classifier to achieve plausible generations. IMAGINE enables the synthesis procedure to simultaneously 1) enforce semantic specificity constraints during the synthesis, 2) produce realistic images without generator training, and 3) give users intuitive control over the generation process.
  arXiv  Detail & Related papers  (2021-04-13T02:00:24Z)
• Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network [92.01145655155374]
  We present an unsupervised image enhancement generative network (UEGAN) It learns the corresponding image-to-image mapping from a set of images with desired characteristics in an unsupervised manner. Results show that the proposed model effectively improves the aesthetic quality of images.
  arXiv  Detail & Related papers  (2020-12-30T03:22:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.