Paired Image-to-Image Translation Quality Assessment Using Multi-Method Fusion

• URL: http://arxiv.org/abs/2205.04186v1
• Date: Mon, 9 May 2022 11:05:15 GMT
• Title: Paired Image-to-Image Translation Quality Assessment Using Multi-Method Fusion
• Authors: Stefan Borasinski, Esin Yavuz, S\'ebastien B\'ehuret
• Abstract summary: This paper proposes a novel approach that combines signals of image quality between paired source and transformation to predict the latter's similarity with a hypothetical ground truth. We trained a Multi-Method Fusion (MMF) model via an ensemble of gradient-boosted regressors to predict Deep Image Structure and Texture Similarity (DISTS) Analysis revealed the task to be feature-constrained, introducing a trade-off at inference between metric time and prediction accuracy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: How best to evaluate synthesized images has been a longstanding problem in image-to-image translation, and to date remains largely unresolved. This paper proposes a novel approach that combines signals of image quality between paired source and transformation to predict the latter's similarity with a hypothetical ground truth. We trained a Multi-Method Fusion (MMF) model via an ensemble of gradient-boosted regressors using Image Quality Assessment (IQA) metrics to predict Deep Image Structure and Texture Similarity (DISTS), enabling models to be ranked without the need for ground truth data. Analysis revealed the task to be feature-constrained, introducing a trade-off at inference between metric computation time and prediction accuracy. The MMF model we present offers an efficient way to automate the evaluation of synthesized images, and by extension the image-to-image translation models that generated them.

Related papers

• MMAR: Towards Lossless Multi-Modal Auto-Regressive Probabilistic Modeling [64.09238330331195]
  We propose a novel Multi-Modal Auto-Regressive (MMAR) probabilistic modeling framework. Unlike discretization line of method, MMAR takes in continuous-valued image tokens to avoid information loss. We show that MMAR demonstrates much more superior performance than other joint multi-modal models.
  arXiv  Detail & Related papers  (2024-10-14T17:57:18Z)
• Attention Down-Sampling Transformer, Relative Ranking and Self-Consistency for Blind Image Quality Assessment [17.04649536069553]
  No-reference image quality assessment is a challenging domain that addresses estimating image quality without the original reference. We introduce an improved mechanism to extract local and non-local information from images via different transformer encoders and CNNs. A self-consistency approach to self-supervision is presented, explicitly addressing the degradation of no-reference image quality assessment (NR-IQA) models.
  arXiv  Detail & Related papers  (2024-09-11T09:08:43Z)
• Semantic Approach to Quantifying the Consistency of Diffusion Model Image Generation [0.40792653193642503]
  We identify the need for an interpretable, quantitative score of the repeatability, or consistency, of image generation in diffusion models. We propose a semantic approach, using a pairwise mean CLIP score as our semantic consistency score.
  arXiv  Detail & Related papers  (2024-04-12T20:16:03Z)
• InVA: Integrative Variational Autoencoder for Harmonization of Multi-modal Neuroimaging Data [3.792342522967013]
  This article proposes a novel approach, referred to as Integrative Variational Autoencoder (textttInVA) method, which borrows information from multiple images obtained from different sources to draw predictive inference of an image. Numerical results demonstrate substantial advantages of textttInVA over VAEs, which typically do not allow borrowing information between input images.
  arXiv  Detail & Related papers  (2024-02-05T05:26:17Z)
• Corner-to-Center Long-range Context Model for Efficient Learned Image Compression [70.0411436929495]
  In the framework of learned image compression, the context model plays a pivotal role in capturing the dependencies among latent representations. We propose the textbfCorner-to-Center transformer-based Context Model (C$3$M) designed to enhance context and latent predictions. In addition, to enlarge the receptive field in the analysis and synthesis transformation, we use the Long-range Crossing Attention Module (LCAM) in the encoder/decoder.
  arXiv  Detail & Related papers  (2023-11-29T21:40:28Z)
• Uncertainty Estimation in Contrast-Enhanced MR Image Translation with Multi-Axis Fusion [6.727287631338148]
  We propose a novel model uncertainty quantification method, Multi-Axis Fusion (MAF) The proposed approach is applied to the task of synthesizing contrast enhanced T1-weighted images based on native T1, T2 and T2-FLAIR scans.
  arXiv  Detail & Related papers  (2023-11-20T20:09:48Z)
• ARNIQA: Learning Distortion Manifold for Image Quality Assessment [28.773037051085318]
  No-Reference Image Quality Assessment (NR-IQA) aims to develop methods to measure image quality in alignment with human perception without the need for a high-quality reference image. We propose a self-supervised approach named ARNIQA for modeling the image distortion manifold to obtain quality representations in an intrinsic manner.
  arXiv  Detail & Related papers  (2023-10-20T17:22:25Z)
• Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation [51.37092275604371]
  Multimodal machine translation (MMT) simultaneously takes the source sentence and a relevant image as input for translation. Recent studies suggest utilizing powerful text-to-image generation models to provide image inputs. However, synthetic images generated by these models often follow different distributions compared to authentic images.
  arXiv  Detail & Related papers  (2023-10-20T09:06:30Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.