ML-CLIPSim: Multi-Layer CLIP Similarity for Machine-Oriented Image Quality

• URL: http://arxiv.org/abs/2605.09479v1
• Date: Sun, 10 May 2026 11:19:02 GMT
• Title: ML-CLIPSim: Multi-Layer CLIP Similarity for Machine-Oriented Image Quality
• Abstract summary: We formulate machine-oriented quality as a latent machine utility and approximate it through pairwise predictive-consistency comparisons.<n>We propose ML-CLIPSim, a differentiable quality metric built on a frozen CLIP visual encoder.
• Score: 13.87968279236735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study full-reference image quality assessment from a machine-centric perspective, where images are evaluated by how well they preserve information for downstream models. We formulate machine-oriented quality as a latent machine utility and approximate it through pairwise predictive-consistency comparisons. To this end, we construct PCMP, a dataset of PSNR-matched distortion pairs labeled by consistency votes from multiple pretrained models. We further propose ML-CLIPSim, a differentiable quality metric built on a frozen CLIP visual encoder, which aggregates intermediate patch-token similarities and global image embeddings. Experiments on machine-preference benchmarks, human-IQA datasets, and learned image compression show that ML-CLIPSim better aligns with machine-oriented preferences than conventional fidelity and perceptual metrics, while remaining competitive for human quality prediction. Used as a compression distortion term, it improves rate--task trade-offs across multiple downstream tasks.

Related papers

• Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment [48.882448108261826]
  Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception.<n>Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores.<n>We introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score.
  arXiv  Detail & Related papers  (2026-05-07T21:38:09Z)
• Hybrid Image Resolution Quality Metric (HIRQM):A Comprehensive Perceptual Image Quality Assessment Framework [0.0]
  We propose the Hybrid Image Resolution Quality Metric (HIRQM) to integrate statistical, multi-scale, and deep learning methods for a comprehensive quality evaluation.<n>A dynamic weighting mechanism adapts component contributions based on image characteristics like brightness and variance, enhancing flexibility across distortion types.<n> evaluated on TID2013 and LIVE datasets, HIRQM Pearson and Spearman correlations of 0.92 and 0.90, outperforming traditional metrics.
  arXiv  Detail & Related papers  (2025-05-04T06:14:10Z)
• Predicting Satisfied User and Machine Ratio for Compressed Images: A Unified Approach [58.71009078356928]
  We create a deep learning-based model to predict Satisfied User Ratio (SUR) and Satisfied Machine Ratio (SMR) of compressed images simultaneously.<n> Experimental results indicate that the proposed model significantly outperforms state-of-the-art SUR and SMR prediction methods.
  arXiv  Detail & Related papers  (2024-12-23T11:09:30Z)
• Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare [99.57567498494448]
  We introduce Compare2Score, an all-around LMM-based no-reference IQA model. During training, we generate scaled-up comparative instructions by comparing images from the same IQA dataset. Experiments on nine IQA datasets validate that the Compare2Score effectively bridges text-defined comparative levels during training.
  arXiv  Detail & Related papers  (2024-05-29T17:26:09Z)
• Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment [49.36799270585947]
  No-reference point cloud quality assessment (NR-PCQA) aims to automatically evaluate the perceptual quality of distorted point clouds without available reference. We propose a novel contrastive pre-training framework tailored for PCQA (CoPA) Our method outperforms the state-of-the-art PCQA methods on popular benchmarks.
  arXiv  Detail & Related papers  (2024-03-15T07:16:07Z)
• Mixing Histopathology Prototypes into Robust Slide-Level Representations for Cancer Subtyping [19.577541771516124]
  Whole-slide image analysis via the means of computational pathology often relies on processing tessellated gigapixel images with only slide-level labels available. Applying multiple instance learning-based methods or transformer models is computationally expensive as each image, all instances have to be processed simultaneously. TheMixer is an under-explored alternative model to common vision transformers, especially for large-scale datasets.
  arXiv  Detail & Related papers  (2023-10-19T14:15:20Z)
• Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training [55.12082817901671]
  We propose a new self-supervised pre-training approach, named Masked and Permuted Vision Transformer (MaPeT)<n>MaPeT employs autoregressive and permuted predictions to capture intra-patch dependencies.<n>Our results demonstrate that MaPeT achieves competitive performance on ImageNet, compared to baselines and competitors under the same model setting.
  arXiv  Detail & Related papers  (2023-06-12T18:12:19Z)
• Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling [66.56355316611598]
  Satisfied Machine Ratio (SMR) is a metric that evaluates the perceptual quality of compressed images and videos for machines. SMR enables perceptual coding for machines and propels Video Coding for Machines from specificity to generality.
  arXiv  Detail & Related papers  (2022-11-13T03:16:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.