GLEAN: Grounded Lightweight Evaluation Anchors for Contamination-Aware Tabular Reasoning

• URL: http://arxiv.org/abs/2603.02212v1
• Date: Thu, 22 Jan 2026 13:56:19 GMT
• Title: GLEAN: Grounded Lightweight Evaluation Anchors for Contamination-Aware Tabular Reasoning
• Abstract summary: We propose a lightweight evaluation protocol that integrates contamination-aware probes, weak-supervision governance, retrieval-reasoning diagnostics, and structured error attribution.<n>We evaluate across TabFact, WTQ via Squall, TableBench, RobuT, and SciTab under a 16GB GPU budget.
• Score: 0.5414847001704249
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tabular reasoning benchmarks mix semantic inference, numerical computation, and brittle table formatting, yet evaluations for small models remain vulnerable to contamination, dataset artifacts, and retrieval failures. We propose GLEAN, a lightweight evaluation protocol that integrates contamination-aware probes, weak-supervision governance, retrieval-reasoning diagnostics, and structured error attribution under tight hardware constraints. We evaluate across TabFact, WTQ via Squall, TableBench, RobuT, and SciTab under a 16GB GPU budget. Using Squall gold SQL as an executable anchor (95.2% execution), GLEAN assigns a deterministic error taxonomy (L0-L4 plus L0.5 context miss) and reveals a stable error-mode separation: TAPEX errors skew toward grounding (L3) while TAPAS errors skew toward hallucination/abstention (L2/L0). We validate evidence-row heuristics against SQL-derived rows on simple queries (0.62 precision / 0.71 recall; hybrid recall 0.81) and show that retrieval Recall@K can saturate even when end-to-end EM/F1 remains limited, motivating attribution beyond raw recall. We release a modular framework with audits and sensitivity checks to make small-model tabular evaluation more contamination-aware and diagnostic.

Related papers

• From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing [50.70862449978062]
  Recent documents achieve table TEDS scores above 93 on OmniDocBench v1.6, yet community feedback and our audit reveal persistent failures on complex real-world tables.<n>To quantify this gap, we introduce TableParseMap, a diagnostic benchmark of 916 real-world tables organized into five challenging scenarios and nine failure types.
  arXiv  Detail & Related papers  (2026-08-10T16:59:30Z)
• Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements [33.97676862545625]
  FinIndices is a large-scale benchmark evaluating data-processing fidelity over uncropped financial statements.<n>Our evaluation reveals two severe LLM vulnerabilities. First, a "Knowledge Bottleneck": despite memorizing formulas during pre-training, models demonstrate fragile pattern matching.<n>Second, a " Structural Bottleneck": the intense cognitive load of generating multi-metric, multi-period tables actively drains reasoning capacity.
  arXiv  Detail & Related papers  (2026-07-22T06:53:07Z)
• Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches [18.727291516223115]
  Desc2Fix is a framework for measuring semantic alignment between bug reports, triggering tests, and developer-written fixes.<n>We analyze 2,857 report-test-patch triplets from Defects4J and SWT-Bench using GPT-4o and DeepSeek-Chat.
  arXiv  Detail & Related papers  (2026-07-20T22:32:23Z)
• Constraint-Anchored Reasoning Traces [9.39195684989942]
  Autoregressive multimodal large language models (MLLMs) suffer from error snowballing.<n>In state-of-the-art open-source MLLMs, once the first error occurs, the reasoning cascades into failure across all remaining steps in 65% of such cases.<n>We propose Constraint-Anchored Reasoning Traces (CART), a neuro-symbolic framework that trains MLLMs to interleave natural language reasoning steps with symbolic constraint assertions.
  arXiv  Detail & Related papers  (2026-07-18T09:24:45Z)
• What Predicts Correctness in Text-to-SQL? A Selective-Prediction Study [0.0]
  We use AUROC to measure how well each ranks correct queries above incorrect ones.<n>On BIRD and Spider, black-box signals such as string, structural, and execution self-consistency, a schema-relevance score, and query executability all fall between about 0.61 and 0.68 AUROC.<n>We ask whether a verifier can be trained. Fine-tuned verifiers, both encoder and generative, reach about 0.77 to 0.79 AUROC in-distribution but fall to about 0.66 on unseen schemas.
  arXiv  Detail & Related papers  (2026-07-07T20:52:58Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Fundamental Limits of Black-Box Safety Evaluation: Information-Theoretic and Computational Barriers from Latent Context Conditioning [1.9290392443571385]
  Black-box safety evaluation of AI systems assumes model behavior on test distributions reliably predicts deployment performance.<n>We formalize and challenge this assumption through latent context-conditioned policies.<n>We establish fundamental limits showing that no black-box evaluator can reliably estimate deployment risk.
  arXiv  Detail & Related papers  (2026-02-19T01:03:11Z)
• SpatialBench-UC: Uncertainty-Aware Evaluation of Spatial Prompt Following in Text-to-Image Generation [0.0]
  SpatialBench-UC is a small, reproducible benchmark for pairwise spatial relations.<n>We release a benchmark package, versioned prompts, pinned configs, per-sample checker outputs, and report tables.<n>We evaluate three baselines, Stable Diffusion 1.5, SD 1.5 BoxDiff, and SD 1.4 GLIGEN.
  arXiv  Detail & Related papers  (2026-01-19T23:37:10Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• TEN: Table Explicitization, Neurosymbolically [13.178005058859583]
  We present a neurosymbolic approach, TEN, for extracting data from text input.<n>Our experiments demonstrate that TEN significantly outperforms purely neurals across multiple datasets and metrics.<n>A 21-participant user study confirms that TEN's tables are rated significantly more accurate.
  arXiv  Detail & Related papers  (2025-08-12T20:16:41Z)
• Data Leakage and Redundancy in the LIT-PCBA Benchmark [0.0]
  LIT-PCBA is widely used to benchmark virtual screening models.<n>We find extensive data leakage and molecular redundancy across its models.<n>Almost all published results on LIT-PCBA are undermined.
  arXiv  Detail & Related papers  (2025-07-29T00:23:45Z)
• Long-Form Information Alignment Evaluation Beyond Atomic Facts [60.25969380388974]
  We introduce MontageLie, a benchmark that constructs deceptive narratives by "montaging" truthful statements without introducing explicit hallucinations.<n>We propose DoveScore, a novel framework that jointly verifies factual accuracy and event-order consistency.
  arXiv  Detail & Related papers  (2025-05-21T17:46:38Z)
• BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics [70.52570641514146]
  We present a benchmark of unfaithful minimal pairs (BUMP) BUMP is a dataset of 889 human-written, minimally different summary pairs. Unlike non-pair-based datasets, BUMP can be used to measure the consistency of metrics.
  arXiv  Detail & Related papers  (2022-12-20T02:17:30Z)
• TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [80.38130122127882]
  TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE) In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled.
  arXiv  Detail & Related papers  (2020-04-30T15:07:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.