Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering

• URL: http://arxiv.org/abs/2603.28583v1
• Date: Mon, 30 Mar 2026 15:32:24 GMT
• Title: Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering
• Abstract summary: We present ChartCynics, a framework designed to unmask visual deception via a "skeptical" reasoning paradigm.<n>Agentic Summarizer optimized via a two-stage protocol: Oracle-Informed SFT for reasoning distillation and Deception-Aware GRPO for adversarial alignment.<n>We show that ChartCynics achieves 74.43% and 64.55% accuracy, providing an absolute performance boost of 29% over the Qwen3-VL-8B backbone.
• Score: 40.87166294341564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the success of Vision-Language Models (VLMs), misleading charts remain a significant challenge due to their deceptive visual structures and distorted data representations. We present ChartCynics, an agentic dual-path framework designed to unmask visual deception via a "skeptical" reasoning paradigm. Unlike holistic models, ChartCynics decouples perception from verification: a Diagnostic Vision Path captures structural anomalies (e.g., inverted axes) through strategic ROI cropping, while an OCR-Driven Data Path ensures numerical grounding. To resolve cross-modal conflicts, we introduce an Agentic Summarizer optimized via a two-stage protocol: Oracle-Informed SFT for reasoning distillation and Deception-Aware GRPO for adversarial alignment. This pipeline effectively penalizes visual traps and enforces logical consistency. Evaluations on two benchmarks show that ChartCynics achieves 74.43% and 64.55% accuracy, providing an absolute performance boost of ~29% over the Qwen3-VL-8B backbone, outperforming state-of-the-art proprietary models. Our results demonstrate that specialized agentic workflows can grant smaller open-source models superior robustness, establishing a new foundation for trustworthy chart interpretation.

Related papers

• REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models [6.014442477518621]
  We introduce REChart, a two-stage training framework that provides process-level supervision over intermediate reasoning steps.<n>First, we synthesize 200k high-quality reasoning trajectories for supervised fine-tuning from a large image-instruction-code pool.<n>Second, we optimize the model via reinforcement learning with two complementary rewards: a emphfidelity reward evaluating code correctness, visual fidelity, and structural consistency, and an emphefficiency reward that assigns each rollout a random thinking budget.
  arXiv  Detail & Related papers  (2026-08-18T06:26:45Z)
• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model [73.59026525350808]
  We propose a dual adversarial fine-tuning framework that jointly optimize visual and semantic supervision signals from two modalities.<n>Our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering tasks.
  arXiv  Detail & Related papers  (2026-07-21T10:49:29Z)
• Patch-Effect Graph Kernels for LLM Interpretability [0.0]
  Mechanistic interpretability aims to reverse-engineer transformer computations by identifying causal circuits through activation patching.<n>We propose a framework that reframes mechanistic analysis as a graph machine-learning problem by representing activation-patching profiles as patch-effect graphs over model components.
  arXiv  Detail & Related papers  (2026-05-07T16:03:47Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Chart Specification: Structural Representations for Incentivizing VLM Reasoning in Chart-to-Code Generation [11.18352269863283]
  Vision-Language Models (VLMs) have shown promise in generating plotting code from chart images.<n>Existing approaches largely rely on supervised fine-tuning, encouraging surface-level token imitation.<n>We propose Chart Specification, a structured intermediate representation that shifts training from text imitation to semantically grounded supervision.
  arXiv  Detail & Related papers  (2026-02-11T14:08:06Z)
• ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch [57.01439313241121]
  We introduce Rollout Posterior Entropy (RPE), a novel metric that quantifies chart complexity.<n>We also develop truth-anchored inverse QA synthesis to guarantee reasoning rigor.<n>To further elevate difficulty and reasoning depth, we filter samples based on model fail-rate and distill high-quality Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2026-01-20T05:11:44Z)
• Losing the Plot: How VLM responses degrade on imperfect charts [11.071294641688496]
  Vision language models (VLMs) show strong results on chart understanding, yet existing benchmarks assume clean figures and fact based queries.<n>We evaluate ChatGPT 4o, Claude Sonnet 4, and Gemini 2.5 Pro, finding sharp performance drops under corruption or occlusion.<n>A key innovation is prompt reverse inconsistency, where models contradict themselves when asked to confirm versus deny the same statement.
  arXiv  Detail & Related papers  (2025-09-22T21:12:20Z)
• Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagrams [0.81996963503528]
  Large Language Models (LLMs) and their multimodal variants (LVLMs) hold immense promise for scientific and engineering applications.<n>Existing evaluation benchmarks largely overlook this challenge, leaving the robust reasoning capabilities of LVLMs underexplored.<n>We introduce the Robust Diagram Reasoning (RDR) framework, a novel approach designed to enhance and rigorously evaluate LVLMs' performance under such conditions.
  arXiv  Detail & Related papers  (2025-08-23T09:50:58Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)
• RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation [50.35403070279804]
  3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. We propose RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction.
  arXiv  Detail & Related papers  (2023-12-19T03:39:56Z)
• Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identification [69.45543438974963]
  We find graph-based methods in the visible-infrared person re-identification task (VI-ReID) suffer from bad generalization because of two issues. The well-trained input features weaken the learning of graph topology, making it not generalized enough during the inference process. We propose a Counterfactual Intervention Feature Transfer (CIFT) method to tackle these problems.
  arXiv  Detail & Related papers  (2022-08-01T16:15:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.