Evidence Absence Is Not Evidence Insufficiency: Diagnosing NEI Construction Artifacts in Fact Verification

• URL: http://arxiv.org/abs/2605.26663v1
• Date: Tue, 26 May 2026 07:51:33 GMT
• Title: Evidence Absence Is Not Evidence Insufficiency: Diagnosing NEI Construction Artifacts in Fact Verification
• Abstract summary: We introduce NEI-CAP, a construction-aware diagnostic protocol for insufficient-evidence evaluation.<n>We instantiate the protocol in SciFact-style scientific verification, with FEVER and HoVer as bounded external controls.
• Score: 6.604874054866016
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evidence absence is not evidence insufficiency, but fact verification benchmarks can make them observationally similar. The Not Enough Information (NEI) label is often operationalized through different evidence conditions, and that choice silently determines what a verifier learns and what its score can hide. We introduce NEI-CAP, a construction-aware diagnostic protocol for insufficient-evidence evaluation. Each NEI example carries the construction family that produced it; NEI-CAP audits shortcut cues, validates hard cases through human adjudication, and tests whether competence transfers across constructions. We instantiate the protocol in SciFact-style scientific verification, with FEVER and HoVer as bounded external controls. Across these settings, NEI competence does not transfer reliably: models trained on shortcut-prone constructions fail to recognize semantically related insufficient evidence, and mixed-construction training narrows but does not close the gap. Fixed-claim diagnostics further show that the evidence condition shifts confidence in the reference Support/Refute label, not only NEI recall, so an aggregate NEI score can hide which problem a model has actually solved.

Related papers

• ProofPulse: Interactive Proof Coverage Analysis for Dafny [47.23158426199152]
  Deductive verification ensures that an implementation satisfies its specification, but successful verification does not guarantee the quality of the specification.<n>We present proofpulse, a tool for Dafny that diagnoses specification quality using a three-valued proof coverage model.
  arXiv  Detail & Related papers  (2026-08-31T14:00:31Z)
• Beyond Relevance: Bayesian Evidence Acquisition for Agentic Whole-Slide Image Reasoning [72.86819994769634]
  Whole-slide image (WSI) reasoning requires an agent to sequentially acquire visual evidence before answering a diagnostic question.<n>We propose BEACON, a plug-and-play agentic framework that reformulates WSI reasoning as a Bayesian evidence acquisition problem.
  arXiv  Detail & Related papers  (2026-08-06T08:40:57Z)
• From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents [47.678562263387214]
  Research agents increasingly conduct multi-round machine-learning experiments in industrial recommendation settings.<n>Yet a completed trajectory is not automatically evidence: generated artifacts may be unsupported or incomplete, executed rounds may be invalid or confounded, and later modifications may obscure earlier findings.<n>We introduce an evidence-grounded framework that couples bounded verification of consequential artifacts with post-execution claim qualification.
  arXiv  Detail & Related papers  (2026-08-05T13:37:38Z)
• EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence [48.314554375516366]
  This paper introduces the Evidence-Grounded Video Question Answering Benchmark (EG-VQA)<n> EG-VQA is comprised of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations.<n>We show that even strong proprietary models struggle to accurately ground their predictions.<n>We propose EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision.
  arXiv  Detail & Related papers  (2026-06-23T16:49:28Z)
• JustDiag!: A Diagnostic Justification Engine for Accountable Root Cause Analysis [10.061964398830142]
  We show that JustDiag maintains an explicit process state over evidence, findings, competing hypotheses, conflicts, and next checks.<n>Relative to a matched control without diagnostic justification, JustDiag achieved stronger outcome and process scores.
  arXiv  Detail & Related papers  (2026-06-17T14:19:17Z)
• Diagnosing Evidence Utilization in Long-Context and Retrieval-Augmented Language Models under Matched Evidence Conditions [0.0]
  A model may answer from parametric priors, fail to use evidence that is present, or cite relevant text without converting it into the final answer.<n>This paper introduces a four-condition diagnostic protocol for evidence-utilization evaluation.
  arXiv  Detail & Related papers  (2026-06-04T22:44:57Z)
• AuthTrace: Diagnosing Evidence Construction in Thematically Dense Single-Author Corpora [6.956097396264084]
  We introduce AuthTrace, a diagnostic benchmark built on thematically dense single-author corpora.<n>AuthTrace provides explicit quoted evidence, exact fan-in annotation, and a unified pack-level protocol measuring evidence recall, evidence precision, and answer correctness.
  arXiv  Detail & Related papers  (2026-05-25T03:10:52Z)
• Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction [4.273094752480624]
  We introduce prover-verifier deliberation (PVD), an inference-time protocol grounded in interactive proof theory.<n>PVD produces both an answer and a structured confidence verdict, allowing a system to report high-confidence answers while abstaining on uncertain cases.
  arXiv  Detail & Related papers  (2026-05-24T15:23:27Z)
• When Verification Fails: How Compositionally Infeasible Claims Escape Rejection [30.404615085122046]
  We show that existing verification benchmarks cannot distinguish between rigorous claim verification and simple salient-constraint reliance.<n>We construct compositionally infeasible claims where the salient constraint is supported but a non-salient constraint is contradicted.<n>We show that different models occupy distinct positions on a shared ROC curve, indicating that the gap between model families reflects differences in verification threshold rather than underlying reasoning ability.
  arXiv  Detail & Related papers  (2026-04-13T04:48:20Z)
• Case-Grounded Evidence Verification: A Framework for Constructing Evidence-Sensitive Supervision [1.7044563989708406]
  We introduce case-grounded evidence verification.<n>We train a standard verifier on the resulting support task.<n>Results suggest that a major bottleneck in evidence grounding is not only model capacity, but the lack of supervision.
  arXiv  Detail & Related papers  (2026-04-10T17:55:38Z)
• VIBEPASS: Can Vibe Coders Really Pass the Vibe Check? [46.85901599242161]
  We evaluate two coupled tasks: emphFault-Triggering Test Generation (FT-Test) and emphFault-targeted Program Repair (FPR)<n>We find that fault-targeted reasoning does not scale with general coding ability.<n>Test-guided repair reveals a complementary insight: when self-generated tests successfully witness a fault, the resulting repair matches or outperforms repair guided by externally provided tests, but tests that fail to witness the fault actively degrade repair below unguided baselines.
  arXiv  Detail & Related papers  (2026-03-16T21:14:28Z)
• Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know" [47.930782177987446]
  Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations.<n>We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks.<n>Because factual knowledge is stable while hallucinations are agreement, cross-regime provides a precise signal of internal uncertainty.
  arXiv  Detail & Related papers  (2026-02-04T18:39:58Z)
• Towards Real-Time Fake News Detection under Evidence Scarcity [66.58597356379907]
  We propose Evaluation-Aware Selection of Experts (EASE), a novel framework for real-time fake news detection.<n>EASE adapts its decision-making process according to the assessed sufficiency of available evidence.<n>We introduce RealTimeNews-25, a new benchmark for evaluating model generalization on emerging news with limited evidence.
  arXiv  Detail & Related papers  (2025-10-13T11:11:46Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)
• Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detection [70.57120710151105]
  We provide a more precise definition of the Semantic Space for the ID distribution. We also define the "Tractable OOD" setting which ensures the distinguishability of OOD and ID distributions.
  arXiv  Detail & Related papers  (2024-11-18T03:09:39Z)
• Evidence Tampering and Chain of Custody in Layered Attestations [0.0]
  In distributed systems, trust decisions are made on the basis of integrity evidence generated via remote attestation. We present algorithms for identifying all such tampering opportunities for given evidence as well as tampering "strategies" by which an adversary can modify incriminating evidence without being detected. Our efforts are intended to help protocol designers ensure their protocols reduce evidence tampering opportunities to the smallest, most trustworthy set of components possible.
  arXiv  Detail & Related papers  (2024-01-31T21:54:53Z)
• Unsupervised Continual Anomaly Detection with Contrastively-learned Prompt [80.43623986759691]
  We introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD. The framework equips the UAD with continual learning capability through contrastively-learned prompts. We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation.
  arXiv  Detail & Related papers  (2024-01-02T03:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.