CLPIPS: A Personalized Metric for AI-Generated Image Similarity

• URL: http://arxiv.org/abs/2604.01234v1
• Date: Thu, 26 Mar 2026 04:23:30 GMT
• Title: CLPIPS: A Personalized Metric for AI-Generated Image Similarity
• Abstract summary: We introduce Customized Learned Perceptual Image Patch Similarity (CLPIPS)<n>CLPIPS is a customized extension of LPIPS that adapts a metric's notion of similarity directly to human judgments.<n>We evaluate CLPIPS on a human subject dataset in which participants iteratively regenerate target images and rank generated outputs by perceived similarity.
• Score: 0.31897361244549305
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Iterative prompt refinement is central to reproducing target images with text to image generative models. Previous studies have incorporated image similarity metrics (ISMs) as additional feedback to human users. Existing ISMs such as LPIPS and CLIP provide objective measures of image likeness but often fail to align with human judgments, particularly in context specific or user driven tasks. In this paper, we introduce Customized Learned Perceptual Image Patch Similarity (CLPIPS), a customized extension of LPIPS that adapts a metric's notion of similarity directly to human judgments. We aim to explore whether lightweight, human augmented fine tuning can meaningfully improve perceptual alignment, positioning similarity metrics as adaptive components for human in the loop workflows with text to image tools. We evaluate CLPIPS on a human subject dataset in which participants iteratively regenerate target images and rank generated outputs by perceived similarity. Using margin ranking loss on human ranked image pairs, we fine tune only the LPIPS layer combination weights and assess alignment via Spearman rank correlation and Intraclass Correlation Coefficient. Our results show that CLPIPS achieves stronger correlation and agreement with human judgments than baseline LPIPS. Rather than optimizing absolute metric performance, our work emphasizes improving alignment consistency between metric predictions and human ranks, demonstrating that even limited human specific fine tuning can meaningfully enhance perceptual alignment in human in the loop text to image workflows.

Related papers

• The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric [78.74729751232852]
  We introduce a large-scale dataset of human similarity judgments over image triplets.<n>We fine-tune a vision-language model to produce our Text-Prompted Image Perceptual Similarity metric.<n>TPIPS aligns more closely with human perception and generalizes reliably beyond the training distribution.
  arXiv  Detail & Related papers  (2026-07-20T17:59:51Z)
• Structured Uncertainty Similarity Score (SUSS): Learning a Probabilistic, Interpretable, Perceptual Metric Between Images [3.1296300934639327]
  Perceptual similarity scores that align with human vision are critical for both training and evaluating computer vision models.<n>We introduce the Structured Uncertainty Similarity Score (SUSS); it models each image through a set of perceptual components.<n>The final score is a weighted sum of component log-probabilities with weights learned from human perceptual datasets.
  arXiv  Detail & Related papers  (2025-12-03T11:48:59Z)
• Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Images [0.7499722271664147]
  The Global-Local Image Perceptual Score (GLIPS) is an image metric designed to assess the photorealistic image quality of AI-generated images. Comprehensive tests across various generative models demonstrate that GLIPS consistently outperforms existing metrics like FID, SSIM, and MS-SSIM in terms of correlation with human scores.
  arXiv  Detail & Related papers  (2024-05-15T15:19:23Z)
• ALIP: Adaptive Language-Image Pre-training with Synthetic Caption [78.93535202851278]
  Contrastive Language-Image Pre-training (CLIP) has significantly boosted the performance of various vision-language tasks. The presence of intrinsic noise and unmatched image-text pairs in web data can potentially affect the performance of representation learning. We propose an Adaptive Language-Image Pre-training (ALIP), a bi-path model that integrates supervision from both raw text and synthetic caption.
  arXiv  Detail & Related papers  (2023-08-16T15:19:52Z)
• R-LPIPS: An Adversarially Robust Perceptual Similarity Metric [71.33812578529006]
  We propose the Robust Learned Perceptual Image Patch Similarity (R-LPIPS) metric. R-LPIPS is a new metric that leverages adversarially trained deep features. We demonstrate the superiority of R-LPIPS compared to the classical LPIPS metric.
  arXiv  Detail & Related papers  (2023-07-27T19:11:31Z)
• End-to-End Context-Aided Unicity Matching for Person Re-identification [100.02321122258638]
  We propose an end-to-end person unicity matching architecture for learning and refining the person matching relations. We use the samples' global context relationship to refine the soft matching results and reach the matching unicity through bipartite graph matching. Given full consideration to real-world person re-identification applications, we achieve the unicity matching in both one-shot and multi-shot settings.
  arXiv  Detail & Related papers  (2022-10-20T07:33:57Z)
• Transparent Human Evaluation for Image Captioning [70.03979566548823]
  We develop a rubric-based human evaluation protocol for image captioning models. We show that human-generated captions show substantially higher quality than machine-generated ones. We hope that this work will promote a more transparent evaluation protocol for image captioning.
  arXiv  Detail & Related papers  (2021-11-17T07:09:59Z)
• Clicking Matters:Towards Interactive Human Parsing [60.35351491254932]
  This work is the first attempt to tackle the human parsing task under the interactive setting. Our IHP solution achieves 85% mIoU on the benchmark LIP, 80% mIoU on PASCAL-Person-Part and CIHP, 75% mIoU on Helen with only 1.95, 3.02, 2.84 and 1.09 clicks per class respectively.
  arXiv  Detail & Related papers  (2021-11-11T11:47:53Z)
• Enriching ImageNet with Human Similarity Judgments and Psychological Embeddings [7.6146285961466]
  We introduce a dataset that embodies the task-general capabilities of human perception and reasoning. The Human Similarity Judgments extension to ImageNet (ImageNet-HSJ) is composed of human similarity judgments. The new dataset supports a range of task and performance metrics, including the evaluation of unsupervised learning algorithms.
  arXiv  Detail & Related papers  (2020-11-22T13:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.