The Truth, the Whole Truth, and Nothing but the Truth: Automatic Visualization Evaluation from Reconstruction Quality

• URL: http://arxiv.org/abs/2603.16873v1
• Date: Mon, 26 Jan 2026 19:36:15 GMT
• Title: The Truth, the Whole Truth, and Nothing but the Truth: Automatic Visualization Evaluation from Reconstruction Quality
• Abstract summary: We propose an automated metric that evaluates visualization quality without relying on extensive human-labeled datasets.<n>Specifically, we introduce a method that measures visualization quality by assessing the reconstruction accuracy of the original data from the visualization itself.<n>This reconstruction-based metric provides an autonomous and scalable proxy for thorough human evaluation, facilitating more efficient and reliable AI-driven visualization.
• Score: 2.014871953160343
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in AI enable the automatic generation of visualizations directly from textual prompts using agentic workflows. However, visualizations produced via one-shot generative methods often suffer from insufficient quality, typically requiring a human in the loop to refine the outputs. Human evaluation, though effective, is costly and impractical at scale. To alleviate this problem, we propose an automated metric that evaluates visualization quality without relying on extensive human-labeled datasets. Instead, our approach uses the original underlying data as implicit ground truth. Specifically, we introduce a method that measures visualization quality by assessing the reconstruction accuracy of the original data from the visualization itself. This reconstruction-based metric provides an autonomous and scalable proxy for thorough human evaluation, facilitating more efficient and reliable AI-driven visualization workflows.

Related papers

• Position: Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered [34.408989226550176]
  This position paper argues that the evaluation of modern visual processing systems should no longer be driven primarily by single-metric image quality assessment benchmarks.<n>Rather than rejecting metrics altogether, this paper calls for a rebalancing of evaluation paradigms, advocating a more human-centered, context-aware, and fine-grained approach to assessing the visual models' outcomes.
  arXiv  Detail & Related papers  (2026-02-28T13:24:34Z)
• Q-REAL: Towards Realism and Plausibility Evaluation for AI-Generated Content [71.46991494014382]
  We introduce Q-Real, a novel dataset for fine-grained evaluation of realism and plausibility in AI-generated images.<n>Q-Real consists of 3,088 images generated by popular text-to-image models.<n>We construct Q-Real Bench to evaluate them on two tasks: judgment and grounding with reasoning.
  arXiv  Detail & Related papers  (2025-11-21T02:43:17Z)
• Transparent and Coherent Procedural Mistake Detection [30.540514590818265]
  Procedural mistake detection (PMD) is a challenging problem of classifying whether a human user has successfully executed a task (specified by a procedural text)<n>We extend PMD to require generating visual self-dialog rationales to inform decisions.<n>Given the impressive, mature image understanding capabilities observed in recent vision-and-language models (VLMs), we curate a suitable benchmark dataset for PMD based on individual frames.
  arXiv  Detail & Related papers  (2024-12-16T16:13:55Z)
• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• Model Reprogramming Outperforms Fine-tuning on Out-of-distribution Data in Text-Image Encoders [56.47577824219207]
  In this paper, we unveil the hidden costs associated with intrusive fine-tuning techniques. We introduce a new model reprogramming approach for fine-tuning, which we name Reprogrammer. Our empirical evidence reveals that Reprogrammer is less intrusive and yields superior downstream models.
  arXiv  Detail & Related papers  (2024-03-16T04:19:48Z)
• SelfEval: Leveraging the discriminative nature of generative models for evaluation [30.239717220862143]
  We present an automated way to evaluate the text alignment of text-to-image generative diffusion models.<n>Our method, called SelfEval, uses the generative model to compute the likelihood of real images given text prompts.
  arXiv  Detail & Related papers  (2023-11-17T18:58:16Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)
• Rubric-Specific Approach to Automated Essay Scoring with Augmentation Training [0.1227734309612871]
  We propose a series of data augmentation operations that train and test an automated scoring model to learn features and functions overlooked by previous works. We achieve state-of-the-art performance in the Automated Student Assessment Prize dataset.
  arXiv  Detail & Related papers  (2023-09-06T05:51:19Z)
• Intrinsic Self-Supervision for Data Quality Audits [35.69673085324971]
  Benchmark datasets in computer vision often contain off-topic images, near duplicates, and label errors. In this paper, we revisit the task of data cleaning and formalize it as either a ranking problem, or a scoring problem. We find that a specific combination of context-aware self-supervised representation learning and distance-based indicators is effective in finding issues without annotation biases.
  arXiv  Detail & Related papers  (2023-05-26T15:57:04Z)
• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation [35.8129864412223]
  This paper proposes a standardized and well-defined human evaluation protocol. We experimentally show that the current automatic measures are incompatible with human perception. We provide insights for designing human evaluation experiments reliably and conclusively.
  arXiv  Detail & Related papers  (2023-04-04T14:14:16Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.