Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment

• URL: http://arxiv.org/abs/2605.06969v2
• Date: Mon, 11 May 2026 01:29:26 GMT
• Title: Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment
• Abstract summary: Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception.<n>Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores.<n>We introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score.
• Score: 48.882448108261826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception. However, existing evaluation approaches tend to over-optimize both hand-crafted no-reference statistics and full-reference metrics that treat the source images as pseudo ground truths. Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores, neither leveraging the reasoning of Multimodal Large Language Models (MLLMs) nor encoding per-image perceptual ambiguity in their supervision, but naively introducing MLLMs with discrete one-hot supervision likewise collapses fused images of similar quality into different rating levels. To address this, we introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score, rather than discrete level predictions, enabling fine-grained discrimination among fused images of similar quality. We exploit the agreement among four IVIF-specific sub-dimensions to construct a per-image soft label whose sharpness reflects how consensual the overall judgment is. We further introduce a tripartite objective combining per-image distributional supervision, within-source-pair Thurstone fidelity for method-level ordering, and cross-source-pair Thurstone fidelity for scene-level ordering across scenes. Extensive experiments demonstrate that FuScore achieves state-of-the-art correlation with human visual preferences.

Related papers

• Failure-Informed Image Self-Augmentation for Multimodal Large Language Model Self-Improvement [50.97806700110183]
  Failure-informed Image Self-Augmentation (textbfFISA) is a framework for MLLM self-improvement that constructs augmented images from the model's own failure cases.<n>Our method generates visually challenging yet answer-preserving image complications, verifies their utility through self-examination, and applies dual fidelity filtering to avoid semantic distortion.
  arXiv  Detail & Related papers  (2026-08-04T14:28:42Z)
• DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning [26.88130913151649]
  Image Difference Captioning (IDC) generates language descriptions that precisely identify differences between two images.<n>We introduce DiffCap-Bench, a comprehensive IDC benchmark covering ten distinct difference categories.<n>We also propose an LLM-as-a-Judge evaluation protocol grounded in human-validated Difference Lists.
  arXiv  Detail & Related papers  (2026-05-06T05:12:41Z)
• Causal Disentanglement for Full-Reference Image Quality Assessment [21.96346718260435]
  We propose a novel FR-IQA paradigm based on causal inference and decoupled representation learning.<n>Our approach formulates degradation estimation as a causal disentanglement process guided by intervention on latent representations.<n>Our method achieves highly competitive performance on standard IQA benchmarks across fully supervised, few-label, and label-free settings.
  arXiv  Detail & Related papers  (2026-04-23T13:18:13Z)
• Bridging Human Evaluation to Infrared and Visible Image Fusion [54.71406895277533]
  Infrared and visible image fusion (IVIF) integrates complementary modalities to enhance scene perception.<n>Current methods predominantly focus on optimizing handcrafted losses and objective metrics, often resulting in fusion outcomes that do not align with human visual preferences.<n>We propose a feedback reinforcement framework that bridges human evaluation to infrared and visible image fusion.
  arXiv  Detail & Related papers  (2026-03-04T09:23:57Z)
• MMAR: Towards Lossless Multi-Modal Auto-Regressive Probabilistic Modeling [64.09238330331195]
  We propose a novel Multi-Modal Auto-Regressive (MMAR) probabilistic modeling framework.<n>Unlike discretization line of method, MMAR takes in continuous-valued image tokens to avoid information loss in an efficient way.<n>We also propose a theoretically proven technique that addresses the numerical stability issue and a training strategy that balances the generation and understanding task goals.
  arXiv  Detail & Related papers  (2024-10-14T17:57:18Z)
• DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wild [73.6767681305851]
  Blind image quality assessment (IQA) in the wild presents significant challenges.<n>Given the difficulty in collecting large-scale training data, leveraging limited data to develop a model with strong generalization remains an open problem.<n>Motivated by the robust image perception capabilities of pre-trained text-to-image (T2I) diffusion models, we propose a novel IQA method, diffusion priors-based IQA.
  arXiv  Detail & Related papers  (2024-05-30T12:32:35Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Bridging the Gap between Multi-focus and Multi-modal: A Focused Integration Framework for Multi-modal Image Fusion [5.417493475406649]
  Multi-modal image fusion (MMIF) integrates valuable information from different modality images into a fused one. This paper proposes a MMIF framework for joint focused integration and modalities information extraction. The proposed algorithm can surpass the state-of-the-art methods in visual perception and quantitative evaluation.
  arXiv  Detail & Related papers  (2023-11-03T12:58:39Z)
• Semantic Image Synthesis via Diffusion Models [174.24523061460704]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks.<n>Recent work on semantic image synthesis mainly follows the de facto GAN-based approaches.<n>We propose a novel framework based on DDPM for semantic image synthesis.
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Paired Image-to-Image Translation Quality Assessment Using Multi-Method Fusion [0.0]
  This paper proposes a novel approach that combines signals of image quality between paired source and transformation to predict the latter's similarity with a hypothetical ground truth. We trained a Multi-Method Fusion (MMF) model via an ensemble of gradient-boosted regressors to predict Deep Image Structure and Texture Similarity (DISTS) Analysis revealed the task to be feature-constrained, introducing a trade-off at inference between metric time and prediction accuracy.
  arXiv  Detail & Related papers  (2022-05-09T11:05:15Z)
• Palette: Image-to-Image Diffusion Models [50.268441533631176]
  We introduce Palette, a simple and general framework for image-to-image translation using conditional diffusion models. On four challenging image-to-image translation tasks, Palette outperforms strong GAN and regression baselines. We report several sample quality scores including FID, Inception Score, Classification Accuracy of a pre-trained ResNet-50, and Perceptual Distance against reference images.
  arXiv  Detail & Related papers  (2021-11-10T17:49:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.