DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

• URL: http://arxiv.org/abs/2608.26757v1
• Date: Thu, 27 Aug 2026 07:53:08 GMT
• Title: DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?
• Abstract summary: We introduce DEEPCHART, an expert-annotated benchmark of 1,482 chart-generation instances from real-world scientific papers, financial filings, and ecosystem reports.<n> Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings.
• Score: 54.230123199584305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering. Our benchmark and associated resources are available at https://github.com/tangdouer1005/DeepChart.

Related papers

• Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework [27.45365558629541]
  Existing interactive tools are reliable but tedious, and mixed-initiative systems, while more efficient, lack generalizability.<n>Recent large language models (MLLMs) offer a unified interface for chart interpretation, yet their ability to extract accurate data tables, especially without visible labels, remains unclear.<n>We build a benchmark featuring diverse real-world charts without data labels to evaluate this capability.<n>Results show that, while current MLLMs reliably reconstruct table structures, they struggle with precise value recovery.
  arXiv  Detail & Related papers  (2026-06-29T05:40:35Z)
• VeriGraph: Towards Verifiable Data-Analytic Agents [71.17693022916896]
  LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable.<n>We propose VeriGraph, a traceable neuro-symbolic reasoning framework that enables agents to construct an explicit heterogeneous evidence directed acyclic graph.<n>Our results suggest that explicit evidence-graph construction is a promising path toward verifiable data-analytic agents.
  arXiv  Detail & Related papers  (2026-06-15T11:50:56Z)
• Generating Statistical Charts with Validation-Driven LLM Workflows [0.0]
  Flowway decomposes chart generation into dataset screening, plot proposal, code synthesis, rendering, validation-driven refinement, description generation, and question-answer generation.<n>It treats chart generation as an inspectable process rather than a one-shot prompt-to-code task, retaining each chart with its code, dataset context, description, and question-answer pairs.<n>Results show that chart-syntax questions are nearly saturated, while value extraction, comparison, and reasoning remain more challenging.
  arXiv  Detail & Related papers  (2026-05-01T17:39:00Z)
• ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning [54.86473583610112]
  We propose PointCoT, which integrates reflective interaction into chain-of-thought reasoning in charts.<n>By prompting MLLMs to generate bounding boxes and re-render charts based on location annotations, we establish connections between textual reasoning steps and visual grounding regions.<n>We develop two instruction-tuned models, ChartPointQ2 and ChartPointQ2.5, which outperform state-of-the-art across several chart benchmarks.
  arXiv  Detail & Related papers  (2025-11-29T04:01:55Z)
• Is this chart lying to me? Automating the detection of misleading visualizations [74.26574031329689]
  Misleading visualizations are a potent driver of misinformation on social media and the web.<n>We introduce Misviz, a benchmark of 2,604 real-world visualizations annotated with 12 types of misleaders.<n>We also release Misviz-synth, a synthetic dataset of 81,814 visualizations generated using Matplotlib and based on real-world data tables.
  arXiv  Detail & Related papers  (2025-08-29T14:36:45Z)
• BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning [51.472854950300416]
  We propose BigCharts, a dataset creation pipeline that generates visually diverse chart images.<n>Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity.<n>By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization.
  arXiv  Detail & Related papers  (2025-08-13T13:39:17Z)
• ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answering [12.285453136336507]
  We propose a code-driven framework designed to enable precise, interpretable reasoning over charts.<n>We first train a high-fidelity model to convert diverse chart images into structured ECharts codes.<n>Then, we design a general chart reasoning data synthesis pipeline.<n>Finally, we train the final multimodal model using a combination of supervised fine-tuning and reinforcement learning.
  arXiv  Detail & Related papers  (2025-06-11T18:55:36Z)
• ChartLens: Fine-grained Visual Attribution in Charts [106.44872805609673]
  Post-Hoc Visual Attribution for Charts identifies fine-grained chart elements that validate a given chart-associated response.<n>We propose ChartLens, a novel chart attribution algorithm that uses segmentation-based techniques to identify chart objects.<n>Our evaluations show that ChartLens improves fine-grained attributions by 26-66%.
  arXiv  Detail & Related papers  (2025-05-25T23:17:32Z)
• Chart-to-Text: A Large-Scale Benchmark for Chart Summarization [9.647079534077472]
  We present Chart-to-text, a large-scale benchmark with two datasets and a total of 44,096 charts. We explain the dataset construction process and analyze the datasets.
  arXiv  Detail & Related papers  (2022-03-12T17:01:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.