Does visualization help AI understand data?

• URL: http://arxiv.org/abs/2507.18022v1
• Date: Thu, 24 Jul 2025 01:47:34 GMT
• Title: Does visualization help AI understand data?
• Authors: Victoria R. Li, Johnathan Sun, Martin Wattenberg,
• Abstract summary: Two vision-language models describe synthetic datasets more precisely when raw data is accompanied by a scatterplot.<n>Our results are initial evidence that AI systems, like humans, can benefit from visualization.
• Score: 2.8624636653931614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Charts and graphs help people analyze data, but can they also be useful to AI systems? To investigate this question, we perform a series of experiments with two commercial vision-language models: GPT 4.1 and Claude 3.5. Across three representative analysis tasks, the two systems describe synthetic datasets more precisely and accurately when raw data is accompanied by a scatterplot, especially as datasets grow in complexity. Comparison with two baselines -- providing a blank chart and a chart with mismatched data -- shows that the improved performance is due to the content of the charts. Our results are initial evidence that AI systems, like humans, can benefit from visualization.

Related papers

• In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding [113.17601814293722]
  We introduce ChartScope, an LVLM optimized for in-depth chart comprehension across diverse chart types.<n>We propose an efficient data generation pipeline that synthesizes paired data for a wide range of chart types.<n>We also establish ChartDQA, a new benchmark for evaluating not only question-answering at different levels but also underlying data understanding.
  arXiv  Detail & Related papers  (2025-07-18T18:15:09Z)
• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding.<n>Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• RealCQA-V2 : Visual Premise Proving A Manual COT Dataset for Charts [2.9201864249313383]
  We introduce Visual Premise Proving, a novel task tailored to refine the process of chart question answering. This approach represents a departure from conventional accuracy-based evaluation methods. A model adept at reasoning is expected to demonstrate proficiency in both data retrieval and the structural understanding of charts.
  arXiv  Detail & Related papers  (2024-10-29T19:32:53Z)
• Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way [65.48447317310442]
  Data Formulator 2 (DF2 for short) is an AI-powered visualization system designed to overcome this limitation.<n> DF2 blends graphical user interfaces and natural language inputs to enable users to convey their intent more effectively.<n>To support efficient iteration, DF2 lets users navigate their iteration history and reuse previous designs, eliminating the need to start from scratch each time.
  arXiv  Detail & Related papers  (2024-08-28T20:12:17Z)
• On Pre-training of Multimodal Language Models Customized for Chart Understanding [83.99377088129282]
  This paper explores the training processes necessary to improve MLLMs' comprehension of charts.<n>We introduce CHOPINLLM, an MLLM tailored for in-depth chart comprehension.
  arXiv  Detail & Related papers  (2024-07-19T17:58:36Z)
• AltChart: Enhancing VLM-based Chart Summarization Through Multi-Pretext Tasks [31.414783623207477]
  We introduce the AltChart dataset, comprising 10,000 real chart images, each paired with a comprehensive summary. We propose a new method for pretraining Vision-Language Models (VLMs) to learn fine-grained chart representations. We conduct extensive evaluations of four leading chart summarization models, analyzing how accessible their descriptions are.
  arXiv  Detail & Related papers  (2024-05-22T12:18:52Z)
• Narrating Causal Graphs with Large Language Models [1.437446768735628]
  This work explores the capability of large pretrained language models to generate text from causal graphs. The causal reasoning encoded in these graphs can support applications as diverse as healthcare or marketing. Results suggest users of generative AI can deploy future applications faster since similar performances are obtained when training a model with only a few examples.
  arXiv  Detail & Related papers  (2024-03-11T19:19:59Z)
• Towards Data-centric Graph Machine Learning: Review and Outlook [120.64417630324378]
  We introduce a systematic framework, Data-centric Graph Machine Learning (DC-GML), that encompasses all stages of the graph data lifecycle. A thorough taxonomy of each stage is presented to answer three critical graph-centric questions. We pinpoint the future prospects of the DC-GML domain, providing insights to navigate its advancements and applications.
  arXiv  Detail & Related papers  (2023-09-20T00:40:13Z)
• Towards Automatic Parsing of Structured Visual Content through the Use of Synthetic Data [65.68384124394699]
  We propose a synthetic dataset, containing Structured Visual Content (SVCs) in the form of images and ground truths. We show the usage of this dataset by an application that automatically extracts a graph representation from an SVC image. Our dataset enables the development of strong models for the interpretation of SVCs while skipping the time-consuming dense data annotation.
  arXiv  Detail & Related papers  (2022-04-29T14:44:52Z)
• Chart-to-Text: A Large-Scale Benchmark for Chart Summarization [9.647079534077472]
  We present Chart-to-text, a large-scale benchmark with two datasets and a total of 44,096 charts. We explain the dataset construction process and analyze the datasets.
  arXiv  Detail & Related papers  (2022-03-12T17:01:38Z)
• Dataset Structural Index: Understanding a machine's perspective towards visual data [0.0]
  I show two meta values with which we can get more information over a visual dataset and use it to optimize data, create better architectures, and have an ability to guess which model would work best. In the paper, I show many applications of DSI, one of which is how the same level of accuracy can be achieved with the same model architectures trained over less amount of data.
  arXiv  Detail & Related papers  (2021-10-05T06:40:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.