Cross-Context Verification: Hierarchical Detection of Benchmark Contamination through Session-Isolated Analysis

• URL: http://arxiv.org/abs/2603.21454v2
• Date: Wed, 01 Apr 2026 14:21:41 GMT
• Title: Cross-Context Verification: Hierarchical Detection of Benchmark Contamination through Session-Isolated Analysis
• Abstract summary: Cross-Context Verification (CCV) is a black-box method that solves the same benchmark problem in N independent sessions.<n>On 9 SWE-bench Verified problems (45 trials, Claude Opus 4.6, temperature 0), CCV achieves perfect separation between contaminated and genuine reasoning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM coding benchmarks face a credibility crisis: widespread solution leakage and test quality issues undermine SWE-bench Verified, while existing detection methods--paraphrase consistency, n-gram overlap, perplexity analysis--never directly observe whether a model reasons or recalls. Meanwhile, simply repeating verification degrades accuracy: multi-turn review generates false positives faster than it discovers true errors, suggesting that structural approaches are needed. We introduce Cross-Context Verification (CCV), a black-box method that solves the same benchmark problem in N independent sessions and measures solution diversity, combined with the Hierarchical Cross-Context Architecture (HCCA), a multi-agent analysis framework that prevents confirmation bias through intentional information restriction across specialized analytical roles. On 9 SWE-bench Verified problems (45 trials, Claude Opus 4.6, temperature 0), CCV achieves perfect separation between contaminated and genuine reasoning (Mann-Whitney U=0, p approx 0.012, r = 1.0). Key findings: (1) contamination is binary--models either recall perfectly or not at all; (2) reasoning absence is a perfect discriminator; (3) 33% of prior contamination labels are false positives; (4) HCCA's independent analysis structure discovers contamination-flaw composite cases that single-analyst approaches miss. A pilot experiment extending HCCA to multi-stage verification (Worker to Verifier to Director) yields a negative result--100% sycophantic confirmation--providing further evidence that information restriction, not structural complexity, is the key mechanism. We release all code and data.

Related papers

• Claim-Level Reliability Assessment for Efficient Test-Time Reasoning [9.168249719115845]
  We propose claim-level falsification as a principle for test-time scaling and instantiate it through Claim-Level Reliability Assessment (CLR)<n>CLR reallocates test-time compute from additional solution sampling to targeted verification.
  arXiv  Detail & Related papers  (2026-08-12T12:33:05Z)
• Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs [20.59321114618083]
  We show that single-turn diagnostics systematically overestimate RAG safety, that contradiction is uncorrelated with safe resolution, and that no universal prompt fix exists.<n>The gap between what models recognize and what they do must be measured and closed before retrieval-augmented systems can be trusted in high-stakes settings.
  arXiv  Detail & Related papers  (2026-05-26T15:18:43Z)
• Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction [4.273094752480624]
  We introduce prover-verifier deliberation (PVD), an inference-time protocol grounded in interactive proof theory.<n>PVD produces both an answer and a structured confidence verdict, allowing a system to report high-confidence answers while abstaining on uncertain cases.
  arXiv  Detail & Related papers  (2026-05-24T15:23:27Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• How Independent are Large Language Models? A Statistical Framework for Auditing Behavioral Entanglement and Reweighting Verifier Ensembles [46.63622714488747]
  Shared pretraining data, distillation, and alignment pipelines can induce hidden behavioral dependencies, latent entanglement.<n>In practice, this manifests as correlated reasoning patterns and synchronized failures.<n>We develop a statistical framework for auditing behavioral entanglement among black-box language models.
  arXiv  Detail & Related papers  (2026-04-08T23:32:06Z)
• Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verification [60.18369393468405]
  Existing verifiers usually underperform owing to a lack of domain knowledge and limited calibration.<n>GLEAN compiles expert-curated protocols into trajectory-informed, well-calibrated correctness signals.<n>We empirically validate GLEAN with agentic clinical diagnosis across three diseases from the MIMIC-IV dataset.
  arXiv  Detail & Related papers  (2026-03-03T09:36:43Z)
• A Multi-Agent Framework for Medical AI: Leveraging Fine-Tuned GPT, LLaMA, and DeepSeek R1 for Evidence-Based and Bias-Aware Clinical Query Processing [0.4349324020366305]
  Large language models (LLMs) show promise for healthcare question answering, but clinical use is limited by weak verification, insufficient evidence grounding, and unreliable confidence signalling.<n>We propose a multi-agent medical QA framework that combines complementary LLMs with evidence retrieval, uncertainty estimation, and bias checks to improve answer reliability.
  arXiv  Detail & Related papers  (2026-02-15T14:17:27Z)
• CausalT5K: Diagnosing and Informing Refusal for Trustworthy Causal Reasoning of Skepticism, Sycophancy, Detection-Correction, and Rung Collapse [1.4608214000864057]
  CausalT5K is a diagnostic benchmark of over 5,000 cases across 10 domains.<n>Unlike synthetic benchmarks, CausalT5K embeds causal traps in realistic narratives.<n>Preliminary experiments reveal a Four-Quadrant Control Landscape where static audit policies universally fail.
  arXiv  Detail & Related papers  (2026-02-09T17:36:56Z)
• Detecting Batch Heterogeneity via Likelihood Clustering [0.9668407688201359]
  Batch effects represent a major confounder in genomic diagnostics.<n>We introduce a method that addresses both limitations by clustering samples according to their Bayesian model evidence.<n>Our method achieves superior clustering accuracy compared to standard correlation-based and dimensionality-reduction approaches.
  arXiv  Detail & Related papers  (2026-01-14T01:49:21Z)
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction [58.51530390018909]
  Large Language Model based multi-agent systems excel at collaborative problem solving but remain brittle to cascading errors.<n>We present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction.
  arXiv  Detail & Related papers  (2025-10-16T05:35:37Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Verification Limits Code LLM Training [23.67882363039948]
  Large language models for code generation increasingly rely on synthetic data, where both problem solutions and verification tests are generated by models.<n>In this work, we study how verification design and strategies influence model performance.
  arXiv  Detail & Related papers  (2025-09-25T07:23:30Z)
• Probabilistic Soundness Guarantees in LLM Reasoning Chains [37.440902632372904]
  Autoregressive Reasoning Entailment Stability (ARES) is a probabilistic framework that evaluates each reasoning step based solely on previously-verified premises.<n>ARES achieves state-of-the-art performance across four benchmarks and demonstrates superior robustness on very long synthetic reasoning chains.
  arXiv  Detail & Related papers  (2025-07-17T09:40:56Z)
• Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs [87.79350168490475]
  This work formally defines self-consistent errors and evaluates mainstream detection methods on them.<n>All four types of detection methods significantly struggle to detect self-consistent errors.<n>Motivated by the observation that self-consistent errors often differ across LLMs, we propose a simple but effective cross-model probe.
  arXiv  Detail & Related papers  (2025-05-23T09:18:56Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• Mitigating the Mutual Error Amplification for Semi-Supervised Object Detection [92.52505195585925]
  We propose a Cross Teaching (CT) method, aiming to mitigate the mutual error amplification by introducing a rectification mechanism of pseudo labels. In contrast to existing mutual teaching methods that directly treat predictions from other detectors as pseudo labels, we propose the Label Rectification Module (LRM)
  arXiv  Detail & Related papers  (2022-01-26T03:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.