VeriGraph: Towards Verifiable Data-Analytic Agents
Abstract Overview
This paper studies how to make LLM-based data-analytic agents more verifiable, arguing that standard linear thought-action traces do not preserve auditable links between raw data, computations, and final claims. The authors propose VeriGraph, a neuro-symbolic framework in which the agent incrementally constructs an explicit heterogeneous evidence DAG during execution, linking data artifacts and natural-language claims. In experiments on four data-intensive benchmarks, the method is evaluated both on task performance and on claim grounding recoverable from the exposed evidence artifact.
Novelty
The distinctive contribution is to make evidence-graph construction the agent’s online reasoning interface rather than a post-hoc explanation, with explicit DAG edges connecting raw data, intermediate variables, grounded claims, and derived conclusions. The work also introduces a graph-aware reinforcement-learning objective that supervises provenance and derivational validity in addition to final-answer quality.
Results
Across TableBench, InfiAgent-DABench, DSBench, and DAB-Step Research, VeriGraph-8B achieves the highest overall score among the evaluated baselines while being the only system reported to expose both computational and derivational provenance. It also attains an 87.61% Grounding Rate, indicating substantially stronger claim-level support from the exposed evidence artifact. Ablation results further show that trajectory SFT and graph-aware RL are important for maintaining both task performance and traceability.
Key Points
- VeriGraph represents data-analytic reasoning as an explicit heterogeneous evidence DAG that supports backward tracing from final claims to raw data sources.
- The framework uses three graph-construction primitives—computational expansion, grounding expansion, and derivational expansion—to separate executable provenance from semantic reasoning.
- Empirical evaluation shows strong task performance together with much higher claim grounding, and ablations indicate that explicit graph supervision is important for these gains.