ChartWalker: Benchmarking the Cross-Chart RAG Task

• URL: http://arxiv.org/abs/2606.23997v1
• Date: Mon, 22 Jun 2026 23:07:38 GMT
• Title: ChartWalker: Benchmarking the Cross-Chart RAG Task
• Abstract summary: We introduce ChartWalker, a novel framework for constructing logically challenging cross-chart RAG tasks.<n>We then propose a structure-aware sampling algorithm that synthesizes semantically coherent, multi-hop reasoning paths.<n>Built with this framework, we release ChartWalker-Bench, a comprehensive benchmark spanning diverse domains and cross-chart query types.
• Score: 18.000660058358115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-Chart Retrieval-Augmented Generation (RAG) is critical for complex multi-modal analytical tasks in scientific, business, and political domains. However, existing benchmarks either focus on tables, which are well-structured and textualized, or generate cross-chart questions by simply extracting key points, which often induces lexical overlap between queries and evidence and yields logically inconsistent reasoning chains. To address this, we introduce ChartWalker, a novel framework for constructing challenging cross-chart RAG tasks. ChartWalker features a hierarchical knowledge graph construction method tailored to charts, which organizes entities and relations by granularity to preserve analytical structure. We then propose a structure-aware sampling algorithm that synthesizes semantically coherent, multi-hop reasoning paths, enabling explicit control over query difficulty and granularity for QA generation. Built with this framework, we release ChartWalker-Bench, a comprehensive benchmark spanning diverse domains and cross-chart query types. Extensive evaluations across major RAG paradigms reveal significant performance gaps, underscoring the benchmark's difficulty and utility. Furthermore, we provide ChartWalker-Agent, an agentic baseline to facilitate analysis and inspire future system design.

Related papers

• Use Graph When It Needs: Efficiently and Adaptively Integrating Retrieval-Augmented Generation with Graphs [12.14017207383674]
  Large language models (LLMs) often struggle with knowledge-intensive tasks due to hallucinations and outdated parametric knowledge.<n>Retrieval-Augmented Generation (RAG) addresses this by integrating external corpora, but its effectiveness is limited by fragmented information in unstructured domain documents.<n>GraphRAG emerged to enhance contextual reasoning through structured knowledge graphs, yet paradoxically underperforms vanilla RAG in real-world scenarios.<n>We propose EA-GraphRAG that dynamically integrates RAG and GraphRAG paradigms through syntax-aware complexity analysis.
  arXiv  Detail & Related papers  (2026-02-03T14:26:28Z)
• A Navigational Approach for Comprehensive RAG via Traversal over Proposition Graphs [23.840376380790783]
  ToPG models its knowledge base as a heterogeneous graph of propositions, entities, and passages.<n>ToPG demonstrates strong performance across both accuracy- and quality-based metrics.
  arXiv  Detail & Related papers  (2026-01-08T11:50:40Z)
• Leveraging Spreading Activation for Improved Document Retrieval in Knowledge-Graph-Based RAG Systems [0.0]
  Retrieval-augmented generation (RAG) systems struggle to reliably retrieve and connect the multi-step evidence required for complicated reasoning tasks.<n>Most of the standard RAG frameworks regard all retrieved information as equally reliable, overlooking the varying credibility and interconnected nature of large textual corpora.<n>We propose a novel RAG framework that employs the spreading activation algorithm to retrieve information from a corpus of documents interconnected by automatically constructed knowledge graphs.
  arXiv  Detail & Related papers  (2025-12-17T19:38:35Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• Enrich-on-Graph: Query-Graph Alignment for Complex Reasoning with LLM Enriching [61.824094419641575]
  Large Language Models (LLMs) struggle with hallucinations and factual errors in knowledge-intensive scenarios like knowledge graph question answering (KGQA)<n>We attribute this to the semantic gap between structured knowledge graphs (KGs) and unstructured queries, caused by inherent differences in their focuses and structures.<n>Existing methods usually employ resource-intensive, non-scalable reasoning on vanilla KGs, but overlook this gap.<n>We propose a flexible framework, Enrich-on-Graph (EoG), which leverages LLMs' prior knowledge to enrich KGs, bridge the semantic gap between graphs and queries.
  arXiv  Detail & Related papers  (2025-09-25T06:48:52Z)
• Assemble Your Crew: Automatic Multi-agent Communication Topology Design via Autoregressive Graph Generation [91.17994756436259]
  Multi-agent systems (MAS) based on large language models (LLMs) have emerged as a powerful solution for dealing with complex problems across diverse domains.<n>Existing approaches are fundamentally constrained by their reliance on a template graph modification paradigm with a predefined set of agents and hard-coded interaction structures.<n>We propose ARG-Designer, a novel autoregressive model that operationalizes this paradigm by constructing the collaboration graph from scratch.
  arXiv  Detail & Related papers  (2025-07-24T09:17:41Z)
• ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering [14.468507852394923]
  Chart question answering (CQA) has become a critical multimodal task for evaluating the reasoning capabilities of vision-language models.<n>We introduce ChartMind, a new benchmark designed for complex CQA tasks in real-world settings.<n>We propose a context-aware yet model-agnostic framework, ChartLLM, that focuses on extracting key contextual elements.
  arXiv  Detail & Related papers  (2025-05-29T08:46:03Z)
• Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning [62.640169289390535]
  SPLIT-RAG is a multi-agent RAG framework that addresses the limitations with question-driven semantic graph partitioning and collaborative subgraph retrieval.<n>The innovative framework first create Semantic Partitioning of Linked Information, then use the Type-Specialized knowledge base to achieve Multi-Agent RAG.<n>The attribute-aware graph segmentation manages to divide knowledge graphs into semantically coherent subgraphs, ensuring subgraphs align with different query types.<n>A hierarchical merging module resolves inconsistencies across subgraph-derived answers through logical verifications.
  arXiv  Detail & Related papers  (2025-05-20T06:44:34Z)
• RAG vs. GraphRAG: A Systematic Evaluation and Key Insights [53.83444096699458]
  We systematically evaluate Retrieval-Augmented Generation (RAG) and GraphRAG on text-based benchmarks.<n>Our results highlight the distinct strengths of RAG and GraphRAG across different tasks and evaluation perspectives.
  arXiv  Detail & Related papers  (2025-02-17T02:36:30Z)
• Graph-Based Multimodal Contrastive Learning for Chart Question Answering [11.828192162922436]
  This work introduces a novel joint multimodal scene graph framework that explicitly models the relationships among chart components and their underlying structures.<n>The framework integrates both visual and textual graphs to capture structural and semantic characteristics.<n>A graph contrastive learning strategy aligns node representations across modalities enabling their seamless incorporation into a transformer decoder as soft prompts.
  arXiv  Detail & Related papers  (2025-01-08T06:27:07Z)
• StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding [54.45681512355684]
  Current chart-related tasks focus on either chart perception that extracts information from the visual charts, or chart reasoning given the extracted data.<n>We introduce StructChart, a novel framework that leverages Structured Triplet Representations (STR) to achieve a unified and label-efficient approach.
  arXiv  Detail & Related papers  (2023-09-20T12:51:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.