Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner

• URL: http://arxiv.org/abs/2507.15509v2
• Date: Thu, 07 Aug 2025 06:40:21 GMT
• Title: Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner
• Authors: Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Yufeng Zhong, Lin Ma,
• Abstract summary: We introduce Chart-R1, a chart-domain vision-language model with reinforcement learning fine-tuning to enable complex chart reasoning.<n>To support Chart-R1, we first propose a novel programmatic data technology to generate high-quality step-by-step chart reasoning data.<n>Then we develop a two-stage training strategy: Chart-COT with step-by-step chain-of-thought supervision, and Chart-RFT with numerically sensitive reinforcement fine-tuning.
• Score: 13.465161900684432
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, inspired by OpenAI-o1/o3 and Deepseek-R1, the R1-Style method based on reinforcement learning fine-tuning has received widespread attention from the community. Previous R1-Style methods mainly focus on mathematical reasoning and code intelligence. It is of great research significance to verify their advantages on more general multimodal data. Chart is an important multimodal data type with rich information, which brings important research challenges in complex reasoning. In this work, we introduce Chart-R1, a chart-domain vision-language model with reinforcement learning fine-tuning to enable complex chart reasoning. To support Chart-R1, we first propose a novel programmatic data synthesis technology to generate high-quality step-by-step chart reasoning data covering single- and multi-subcharts, which makes up for the lack of reasoning data in the chart domain. Then we develop a two-stage training strategy: Chart-COT with step-by-step chain-of-thought supervision, and Chart-RFT with numerically sensitive reinforcement fine-tuning. Chart-COT aims to decompose complex chart reasoning tasks into fine-grained, understandable subtasks through step-by-step supervision, which lays a good foundation for improving the reasoning level of reinforcement learning. Chart-RFT utilize the typical group relative policy optimization strategy, in which a relatively soft reward is adopted for numerical response to emphasize the numerical sensitivity in the chart domain. We conduct extensive experiments on open-source benchmarks and self-built chart reasoning dataset (\emph{i.e., ChartRQA}). Experimental results show that Chart-R1 has significant advantages compared to chart-domain methods, even comparable to open/closed source large-scale models (\emph{e.g., GPT-4o, Claude-3.5}).

Related papers

• ProGraph-R1: Progress-aware Reinforcement Learning for Graph Retrieval Augmented Generation [37.11787010202267]
  We propose ProGraph-R1, a progress-aware agentic framework for graph-based retrieval and multi-step reasoning.<n>ProGraph-R1 introduces a structure-aware hypergraph retrieval mechanism that jointly considers semantic relevance and graph connectivity.<n> Experiments on multi-hop question answering benchmarks demonstrate that ProGraph-R1 consistently improves reasoning accuracy and generation quality over existing GraphRAG methods.
  arXiv  Detail & Related papers  (2026-01-25T08:58:44Z)
• ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch [57.01439313241121]
  We introduce Rollout Posterior Entropy (RPE), a novel metric that quantifies chart complexity.<n>We also develop truth-anchored inverse QA synthesis to guarantee reasoning rigor.<n>To further elevate difficulty and reasoning depth, we filter samples based on model fail-rate and distill high-quality Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2026-01-20T05:11:44Z)
• ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learning [64.4193334712998]
  The chart-to-code generation task requires MLLMs to convert chart images into executable code.<n>This task faces two main challenges: limited data diversity and the difficulty of maintaining visual consistency between generated charts and the original ones.<n>We propose ReChartPrompt, leveraging real-world, human-designed charts extracted from arXiv papers as prompts.<n>We also propose ChartSimRL, a GRPO-based reinforcement learning algorithm guided by a novel chart similarity reward.
  arXiv  Detail & Related papers  (2025-08-25T02:32:56Z)
• BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning [51.472854950300416]
  We propose BigCharts, a dataset creation pipeline that generates visually diverse chart images.<n>Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity.<n>By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization.
  arXiv  Detail & Related papers  (2025-08-13T13:39:17Z)
• Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning [20.05893083101089]
  Graph-R1 is an agentic GraphRAG framework via end-to-end reinforcement learning (RL)<n>It introduces lightweight knowledge hypergraph construction, models retrieval as a multi-turn agent-environment interaction.<n>Experiments on standard RAG datasets show that Graph-R1 outperforms traditional GraphRAG and RL-enhanced RAG methods in reasoning accuracy, retrieval efficiency, and generation quality.
  arXiv  Detail & Related papers  (2025-07-29T15:01:26Z)
• ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answering [12.285453136336507]
  We propose a code-driven framework designed to enable precise, interpretable reasoning over charts.<n>We first train a high-fidelity model to convert diverse chart images into structured ECharts codes.<n>Then, we design a general chart reasoning data synthesis pipeline.<n>Finally, we train the final multimodal model using a combination of supervised fine-tuning and reinforcement learning.
  arXiv  Detail & Related papers  (2025-06-11T18:55:36Z)
• Compile Scene Graphs with Reinforcement Learning [69.36723767339001]
  Next-token prediction is the fundamental principle for training large language models (LLMs)<n>We introduce R1-SGG, a multimodal LLM (M-LLM) initially trained via supervised fine-tuning (SFT) on the scene graph dataset.<n>We design a set of graph-centric rewards, including three recall-based variants -- Hard Recall, Hard Recall+Relax, and Soft Recall.
  arXiv  Detail & Related papers  (2025-04-18T10:46:22Z)
• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding.<n>Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• ChartAdapter: Large Vision-Language Model for Chart Summarization [13.499376163294816]
  ChartAdapter is a lightweight transformer module designed to bridge the gap between charts and textual summaries.<n>By integrating ChartAdapter with an LLM, we enable end-to-end training and efficient chart summarization.
  arXiv  Detail & Related papers  (2024-12-30T05:07:34Z)
• ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild [28.643565008567172]
  We introduce ChartGemma, a novel chart understanding and reasoning model developed over PaliGemma. Rather than relying on underlying data tables, ChartGemma is trained on instruction-tuning data generated directly from chart images. Our simple approach achieves state-of-the-art results across $5$ benchmarks spanning chart summarization, question answering, and fact-checking.
  arXiv  Detail & Related papers  (2024-07-04T22:16:40Z)
• TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning [83.58521787193293]
  We present TinyChart, an efficient MLLM for chart understanding with only 3B parameters. TinyChart overcomes two key challenges in efficient chart understanding: (1) reduce the burden of learning numerical computations through a Program-of-Thoughts (PoT) learning strategy, and (2) reduce lengthy vision feature sequences produced by the vision transformer for high-resolution images through a Vision Token Merging module.
  arXiv  Detail & Related papers  (2024-04-25T14:23:24Z)
• ChartThinker: A Contextual Chain-of-Thought Approach to Optimized Chart Summarization [32.19963543411396]
  This study constructs a large-scale dataset of comprehensive chart-caption pairs and fine-tuning instructions on each chart. We propose an innovative chart summarization method, ChartThinker, which synthesizes deep analysis based on chains of thought. Built upon the curated datasets, our trained model consistently exhibits superior performance in chart summarization tasks.
  arXiv  Detail & Related papers  (2024-03-17T14:49:09Z)
• ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning [54.89249749894061]
  ChartAssistant is a vision-language model for universal chart comprehension and reasoning. It undergoes a two-stage training process, starting with pre-training on chart-to-table parsing to align chart and text. Experimental results demonstrate significant performance gains over the state-of-the-art UniChart and Chartllama method.
  arXiv  Detail & Related papers  (2024-01-04T17:51:48Z)
• StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding [54.45681512355684]
  Current chart-related tasks focus on either chart perception that extracts information from the visual charts, or chart reasoning given the extracted data.<n>We introduce StructChart, a novel framework that leverages Structured Triplet Representations (STR) to achieve a unified and label-efficient approach.
  arXiv  Detail & Related papers  (2023-09-20T12:51:13Z)
• ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules [89.75395046894809]
  We present ChartReader, a unified framework that seamlessly integrates chart derendering and comprehension tasks. Our approach includes a transformer-based chart component detection module and an extended pre-trained vision-language model for chart-to-X tasks. Our proposed framework can significantly reduce the manual effort involved in chart analysis, providing a step towards a universal chart understanding model.
  arXiv  Detail & Related papers  (2023-04-05T00:25:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.