DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA

• URL: http://arxiv.org/abs/2605.00905v1
• Date: Wed, 29 Apr 2026 02:34:51 GMT
• Title: DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA
• Abstract summary: Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer.<n>We present DIAGRAMS, a lightweight, schema-driven review framework that decouples interface logic from dataset-specific structures.
• Score: 56.73431446011309
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer, rather than only the region containing the final response. Creating such structured evidence across diagrams, charts, maps, circuits, and infographics is time-consuming, and existing annotation tools tightly couple their interfaces to dataset-specific formats. We present DIAGRAMS, a lightweight, schema-driven review framework that decouples interface logic from dataset-specific JSON structures through an internal meta-schema and dataset adapters. Given an image and QA pair with optional candidate regions, the system performs QA-conditioned evidence selection and proposes the regions required for reasoning. When QA pairs or candidate regions are missing, it generates them and supports human verification and refinement. Across six Diagram QA datasets, model-suggested evidence achieves 85.39% precision and 75.30% recall against reviewer-final selections (micro-averaged). These results indicate that the review-first framework reduces manual region creation while maintaining high agreement with final reasoning-level attributions. We release a public demo and installable package to support dataset auditing, grounded supervision creation, and grounded evaluation.

Related papers

• DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams [54.39165467997251]
  We introduce DRAGON, a benchmark for evaluating evidence-grounded visual reasoning in diagrams.<n>Given a diagram, a question, and the correct answer, a model must predict bounding boxes that correspond to the visual elements required to justify the answer.<n>The DRAGON dataset contains 11,664 annotated question instances collected from six diagram QA datasets.
  arXiv  Detail & Related papers  (2026-04-28T05:24:05Z)
• What Lies Beneath: A Call for Distribution-based Visual Question & Answer Datasets [3.1351527202068445]
  We argue for a dedicated VQA benchmark for scientific charts where there is no 1-to-1 correspondence between chart marks and underlying data.<n>We generate synthetic histogram charts based on ground truth data, and ask both humans and a large reasoning model questions where precise answers depend on access to the underlying data.<n>We release the open-source dataset, including figures, underlying data, distribution parameters used to generate the data, and bounding boxes for all figure marks and text for future research.
  arXiv  Detail & Related papers  (2026-01-29T19:00:01Z)
• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding.<n>Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• Trust but Verify: Programmatic VLM Evaluation in the Wild [62.14071929143684]
  Programmatic VLM Evaluation (PROVE) is a new benchmarking paradigm for evaluating VLM responses to open-ended queries. We benchmark the helpfulness-truthfulness trade-offs of a range ofVLMs on PROVE, finding that very few are in-fact able to achieve a good balance between the two.
  arXiv  Detail & Related papers  (2024-10-17T01:19:18Z)
• OpenCQA: Open-ended Question Answering with Charts [6.7038829115674945]
  We introduce a new task called OpenCQA, where the goal is to answer an open-ended question about a chart with texts. We implement and evaluate a set of baselines under three practical settings. Our analysis of the results show that the top performing models generally produce fluent and coherent text.
  arXiv  Detail & Related papers  (2022-10-12T23:37:30Z)
• Towards Complex Document Understanding By Discrete Reasoning [77.91722463958743]
  Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language. We introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages and 16,558 question-answer pairs. We develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions.
  arXiv  Detail & Related papers  (2022-07-25T01:43:19Z)
• Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking [30.155625852894797]
  We propose a browser-based benchmarking tool for researchers and challenge organizers. Our tool helps test generalization capabilities of models across multiple datasets. Interactive filtering facilitates discovery of problematic behavior.
  arXiv  Detail & Related papers  (2021-10-11T11:08:35Z)
• A Revised Generative Evaluation of Visual Dialogue [80.17353102854405]
  We propose a revised evaluation scheme for the VisDial dataset. We measure consensus between answers generated by the model and a set of relevant answers. We release these sets and code for the revised evaluation scheme as DenseVisDial.
  arXiv  Detail & Related papers  (2020-04-20T13:26:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.