LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models

• URL: http://arxiv.org/abs/2606.02535v1
• Date: Mon, 01 Jun 2026 17:40:09 GMT
• Title: LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models
• Abstract summary: LL-Bench is a benchmark for evaluating the capabilities of large-scale generative models on low-level vision tasks.<n>We present a systematic diagnosis that reveals the performance boundaries and failure unique modes of large-scale generative models.<n>We propose textbfLL-Score, an MLLM-based evaluator that captures both restoration quality and hallucination existence.
• Score: 51.91197140548632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large-scale generative models have demonstrated remarkable capabilities across image generation and editing tasks. However, their performance in low-level vision tasks, which require pixel-wise control, remains insufficiently studied. To address this gap, we introduce \textbf{LL-Bench}, a comprehensive \textbf{Benchmark} for evaluating the capabilities of large-scale generative models on \textbf{L}ow-\textbf{L}evel vision tasks. The benchmark comprises 2,469 real-world degraded images covering 16 low-level degradation tasks, and 28,919 restored images produced by 10 state-of-the-art large-scale generative models and 21 conventional restoration models, which are annotated with 152,020 expert-level pairwise human preferences and 28,334 quality scores. Built upon LL-Bench, we present a systematic diagnosis that reveals the performance boundaries and unique failure modes of large-scale generative models across diverse low-level vision tasks, compared with conventional representative restoration approaches. Moreover, we investigate the effectiveness of current quality evaluation metrics on LL-Bench, which exhibit significant discrepancy with human ratings. To better align restored-image quality assessment with human preferences, we further propose \textbf{LL-Score}, an MLLM-based evaluator that captures both restoration quality and hallucination existence. Extensive experiments demonstrate that LL-score not only outperforms existing image quality assessment metrics, but also serves as a promising reward model for training generative models on low-level vision tasks.

Related papers

• LEIQ-Assessor: Multi-dimensional Quality Assessment of Low-light Enhanced Images via Multi-task Learning [44.13842840233756]
  We present textbfLEIQ-Assessor, a multi-dimensional quality assessment model for low-light image enhancement based on multi-task learning.<n>Our method achieved second place in the QoMEX 2026 Grand Challenge on Low-light Enhanced Image Quality Assessment.
  arXiv  Detail & Related papers  (2026-06-29T03:49:43Z)
• Evaluating the Performance of Open-Vocabulary Object Detection in Low-quality Image [0.0]
  We introduce a new dataset that simulates low-quality images in the real world.<n>We find that although open-vocabulary object detection models exhibited no significant decrease in mAP scores under low-level image degradation, the performance of all models dropped sharply under high-level image degradation.
  arXiv  Detail & Related papers  (2025-12-28T06:18:22Z)
• Evaluating and Preserving High-level Fidelity in Super-Resolution [50.65679806442527]
  Super-Resolution (SR) models are achieving impressive effects in reconstructing details and delivering pleasant visually outputs.<n>However, the overpowering generative ability can sometimes hallucinate and thus change the image content.<n>This type of high-level change can be easily identified by humans yet not well-studied in existing low-level image quality metrics.
  arXiv  Detail & Related papers  (2025-12-07T22:53:34Z)
• Q-REAL: Towards Realism and Plausibility Evaluation for AI-Generated Content [71.46991494014382]
  We introduce Q-Real, a novel dataset for fine-grained evaluation of realism and plausibility in AI-generated images.<n>Q-Real consists of 3,088 images generated by popular text-to-image models.<n>We construct Q-Real Bench to evaluate them on two tasks: judgment and grounding with reasoning.
  arXiv  Detail & Related papers  (2025-11-21T02:43:17Z)
• Mitigating Low-Level Visual Hallucinations Requires Self-Awareness: Database, Model and Training Strategy [53.07517728420411]
  We introduce the first instruction database specifically focused on hallucinations in low-level vision tasks.<n>We propose the Self-Awareness Failure Elimination (SAFEQA) model to improve the perception and comprehension abilities of the model in low-level vision tasks.<n>We conduct comprehensive experiments on low-level vision tasks, with the results demonstrating that our proposed method significantly enhances self-awareness of the model in these tasks and reduces hallucinations.
  arXiv  Detail & Related papers  (2025-03-26T16:05:01Z)
• ReF-LDM: A Latent Diffusion Model for Reference-based Face Image Restoration [11.712490684089609]
  We propose ReF-LDM, an adaptation of LDM designed to generate HQ face images conditioned on one LQ image and multiple HQ reference images.<n>Our model integrates an effective and efficient mechanism, CacheKV, to leverage the reference images during the generation process.<n> Lastly, we construct FFHQ-Ref, a dataset consisting of 20,405 high-quality (HQ) face images with corresponding reference images.
  arXiv  Detail & Related papers  (2024-12-06T13:49:10Z)
• Learning from History: Task-agnostic Model Contrastive Learning for Image Restoration [79.04007257606862]
  This paper introduces an innovative method termed 'learning from history', which dynamically generates negative samples from the target model itself. Our approach, named Model Contrastive Learning for Image Restoration (MCLIR), rejuvenates latency models as negative models, making it compatible with diverse image restoration tasks.
  arXiv  Detail & Related papers  (2023-09-12T07:50:54Z)
• Quality Assessment of Low Light Restored Images: A Subjective Study and an Unsupervised Model [10.119600046984088]
  This work studies the subjective perception of low light restored images and their unsupervised NR QA. We first create a dataset of restored low light images using various LLR methods, conduct a subjective QA study and benchmark the performance of existing QA methods. We then present a self-supervised contrastive learning technique to extract distortion aware features from the restored low light images.
  arXiv  Detail & Related papers  (2022-02-04T18:06:07Z)
• Improved Techniques for Training Score-Based Generative Models [104.20217659157701]
  We provide a new theoretical analysis of learning and sampling from score models in high dimensional spaces. We can effortlessly scale score-based generative models to images with unprecedented resolutions. Our score-based models can generate high-fidelity samples that rival best-in-class GANs on various image datasets.
  arXiv  Detail & Related papers  (2020-06-16T09:17:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.