ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding

• URL: http://arxiv.org/abs/2603.27064v1
• Date: Sat, 28 Mar 2026 00:45:05 GMT
• Title: ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding
• Abstract summary: Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language.<n>We introduce ChartNet, a high-quality, million-scale multimodal dataset designed to advance chart interpretation and reasoning.
• Score: 26.60504788691021
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a capability where current vision-language models (VLMs) remain limited. We introduce ChartNet, a high-quality, million-scale multimodal dataset designed to advance chart interpretation and reasoning. ChartNet leverages a novel code-guided synthesis pipeline to generate 1.5 million diverse chart samples spanning 24 chart types and 6 plotting libraries. Each sample consists of five aligned components: plotting code, rendered chart image, data table, natural language summary, and question-answering with reasoning, providing fine-grained cross-modal alignment. To capture the full spectrum of chart comprehension, ChartNet additionally includes specialized subsets encompassing human annotated data, real-world data, safety, and grounding. Moreover, a rigorous quality-filtering pipeline ensures visual fidelity, semantic accuracy, and diversity across chart representations. Fine-tuning on ChartNet consistently improves results across benchmarks, demonstrating its utility as large-scale supervision for multimodal models. As the largest open-source dataset of its kind, ChartNet aims to support the development of foundation models with robust and generalizable capabilities for data visualization understanding. The dataset is publicly available at https://huggingface.co/datasets/ibm-granite/ChartNet

Related papers

• ChartComplete: A Taxonomy-based Inclusive Chart Dataset [1.9728521995447947]
  multimodal large language models (MLLMs) are proving to be efficient and accurate in understanding charts.<n>To accurately measure the performance of MLLMs, the research community has developed multiple datasets to serve as benchmarks.<n>We present the ChartComplete dataset as is to the community to build upon it.
  arXiv  Detail & Related papers  (2026-01-15T14:51:15Z)
• BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning [51.472854950300416]
  We propose BigCharts, a dataset creation pipeline that generates visually diverse chart images.<n>Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity.<n>By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization.
  arXiv  Detail & Related papers  (2025-08-13T13:39:17Z)
• In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding [113.17601814293722]
  We introduce ChartScope, an LVLM optimized for in-depth chart comprehension across diverse chart types.<n>We propose an efficient data generation pipeline that synthesizes paired data for a wide range of chart types.<n>We also establish ChartDQA, a new benchmark for evaluating not only question-answering at different levels but also underlying data understanding.
  arXiv  Detail & Related papers  (2025-07-18T18:15:09Z)
• ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understanding [14.54762768001729]
  We propose ChartCards, a unified chart-metadata generation framework for multi-task chart understanding.<n>Using ChartCards, we construct MetaChart, a large-scale high-quality dataset containing 10,862 data tables, 85K charts, and 170 K high-quality chart captions.<n>Fine-tuning six different models on MetaChart resulted in an average performance improvement of 5% across all tasks.
  arXiv  Detail & Related papers  (2025-05-21T03:07:47Z)
• Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback [37.275533538711436]
  We propose a hierarchical pipeline and a new dataset for chart generation.<n>Our dataset, Text2Chart31, includes 31 unique plot types referring to the Matplotlib library.<n>We introduce a reinforcement learning-based instruction tuning technique for chart generation tasks without requiring human feedback.
  arXiv  Detail & Related papers  (2024-10-05T07:25:56Z)
• SynChart: Synthesizing Charts from Language Models [50.73888371511983]
  This work explores the potential of using LLMs alone for data generation and develop competitive multi-modality models focusing on chart understanding. We construct a large-scale chart dataset, SynChart, which contains approximately 4 million diverse chart images with over 75 million dense annotations. We trained a 4.2B chart-expert model using this dataset and achieve near-GPT-4O performance on the ChartQA task, surpassing GPT-4V.
  arXiv  Detail & Related papers  (2024-09-25T00:18:12Z)
• ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning [55.22996841790139]
  We benchmark the ability of off-the-shelf Multi-modal Large Language Models (MLLMs) in the chart domain.<n>We construct ChartX, a multi-modal evaluation set covering 18 chart types, 7 chart tasks, 22 disciplinary topics, and high-quality chart data.<n>We develop ChartVLM to offer a new perspective on handling multi-modal tasks that strongly depend on interpretable patterns.
  arXiv  Detail & Related papers  (2024-02-19T14:48:23Z)
• ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning [54.89249749894061]
  ChartAssistant is a vision-language model for universal chart comprehension and reasoning. It undergoes a two-stage training process, starting with pre-training on chart-to-table parsing to align chart and text. Experimental results demonstrate significant performance gains over the state-of-the-art UniChart and Chartllama method.
  arXiv  Detail & Related papers  (2024-01-04T17:51:48Z)
• ChartLlama: A Multimodal LLM for Chart Understanding and Generation [70.1393163657813]
  We create a high-quality instruction-tuning dataset leveraging GPT-4. Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset.
  arXiv  Detail & Related papers  (2023-11-27T15:20:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.