CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis

• URL: http://arxiv.org/abs/2606.19819v1
• Date: Thu, 18 Jun 2026 05:48:37 GMT
• Title: CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis
• Abstract summary: Decomposing compound sentences into atomic, verifiable claims is a prerequisite for reliable automated fact-checking.<n>Prior work has relied on token-overlap (Jaccard) metrics that systematically underestimate decomposition quality for paraphrastic claims.<n>We present Credence, a revised claim decomposition and evaluation framework addressing both shortcomings.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decomposing compound sentences into atomic, verifiable claims is a prerequisite for reliable automated fact-checking. Prior work has relied on token-overlap (Jaccard) metrics that systematically underestimate decomposition quality for paraphrastic claims, and has lacked formal termination analysis for the repair loop. We present Credence, a revised claim decomposition and evaluation framework addressing both shortcomings. Our contributions are: (1) Semantic-F1: we use BGE-large cosine similarity fidelity metric that resolves Jaccard's penalisation and improves downstream fact-checking accuracy; (2) Convergence theorems: we formally characterise four properties of the repair pipeline, establishing that rule-based repair is monotone and finitely terminating under an oracle parser assumption; LLM-based self-repair is provably non-monotone and requires an early-exit guard; (3) Three evaluation benchmarks spanning social-media, encyclopaedic, and news domains for cross-domain generalisation measurement; (4) Multi-model benchmarking across four decomposer models (3.8B-12B) and a closed API model. Experiments on SocialClaimSplit, WikiSplitBench, and ClaimDecompBench show that Semantic-F1 outperforms Jaccard-F1 by +15-32pp. EPR ranges from 0.94 to 1.00 on SocialClaimSplit and WikiSplitBench, while ClaimDecompBench includes lower base EPR cases (down to 0.824) due to harder news-domain constructions, and rule-repair reduces the Atomicity Violation Rate (AVR) by 47-100% relative to the base model without degrading fidelity.

Related papers

• Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches [18.727291516223115]
  Desc2Fix is a framework for measuring semantic alignment between bug reports, triggering tests, and developer-written fixes.<n>We analyze 2,857 report-test-patch triplets from Defects4J and SWT-Bench using GPT-4o and DeepSeek-Chat.
  arXiv  Detail & Related papers  (2026-07-20T22:32:23Z)
• Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis? [44.69722591655738]
  Existing methods for multimodal sentiment analysis under missing modalities usually follow a repair-first paradigm.<n>We propose textbfSufficiency-textbfInformed textbfEvidential textbfValtextbfvE (textbfSIEVE) that turns whether to repair'' into an explicit, learnable decision.
  arXiv  Detail & Related papers  (2026-07-19T14:07:43Z)
• PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception [75.70273182182397]
  PerceptionRubrics is a rubric-based evaluation framework.<n>It addresses the gap between saturated benchmark scores and real-world brittleness.<n>It pairs 1,038 information-dense images with over 10,000 instance-specific rubrics.
  arXiv  Detail & Related papers  (2026-06-26T17:59:15Z)
• A Family of Divergence Measures for Evaluating the Reconstruction Quality of Explainable Ensemble Trees [0.41292255339309664]
  Correlation-based approaches fail to detect systematic discrepancies in co-occurrence structure.<n>We propose a statistical framework grounded in the agreement-association distinction.<n>The framework incorporates four complementary measures capturing distinct structural facets of approximation quality.
  arXiv  Detail & Related papers  (2026-05-19T09:56:04Z)
• Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals [0.0]
  We formalize a taxonomy of four behavioral fraud patterns (P1-P4) covering inter-event timing, burst structure, multi-account graph motifs, and velocity-rule trigger rates.<n>We benchmark CTGAN, TVAE, GaussianCopula, and TabularARGN on IEEE-CIS Fraud Detection and the Amazon Fraud dataset.<n>The P1-P4 framework extends to any domain with entity-level sequential data, including healthcare and network security.
  arXiv  Detail & Related papers  (2026-04-13T19:36:00Z)
• Verify Before You Fix: Agentic Execution Grounding for Trustworthy Cross-Language Code Analysis [0.0]
  We build a unified cross-language vulnerability lifecycle framework for software vulnerability analysis.<n>We achieve 89.84-92.02% intra-language detection accuracy and 74.43-80.12% zero-shot cross-language F1.<n>These results demonstrate that execution-grounded closed-loop reasoning is a principled and practically deployable mechanism for trustworthy LLM-driven agentic AI.
  arXiv  Detail & Related papers  (2026-04-12T20:22:23Z)
• Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation [45.13215113490545]
  GuarantRAG is a framework that explicitly decouples reasoning from evidence integration.<n> Experiments on five QA benchmarks demonstrate that GuarantRAG improves accuracy by up to 12.1%.
  arXiv  Detail & Related papers  (2026-04-09T09:52:21Z)
• How Independent are Large Language Models? A Statistical Framework for Auditing Behavioral Entanglement and Reweighting Verifier Ensembles [46.63622714488747]
  Shared pretraining data, distillation, and alignment pipelines can induce hidden behavioral dependencies, latent entanglement.<n>In practice, this manifests as correlated reasoning patterns and synchronized failures.<n>We develop a statistical framework for auditing behavioral entanglement among black-box language models.
  arXiv  Detail & Related papers  (2026-04-08T23:32:06Z)
• PRECEPT: Planning Resilience via Experience, Context Engineering & Probing Trajectories A Unified Framework for Test-Time Adaptation with Compositional Rule Learning and Pareto-Guided Prompt Evolution [2.28438857884398]
  LLM agents that store knowledge as natural language suffer steep retrieval degradation as condition count grows.<n>We introduce PRECEPT, a unified framework for test-time adaptation with three tightly coupled components.
  arXiv  Detail & Related papers  (2026-03-10T13:16:45Z)
• ACAR: Adaptive Complexity Routing for Multi-Model Ensembles with Auditable Decision Traces [3.151184728006369]
  We present ACAR, a measurement framework for studying multi-model orchestration under auditable conditions.<n>ACAR uses self-consistency variance (sigma) computed from N=3 probe samples to route tasks across single-model, two-model, and three-model execution modes.<n>We evaluate ACAR on 1,510 tasks spanning four benchmarks, producing more than 7,550 auditable runs.
  arXiv  Detail & Related papers  (2026-02-06T23:27:17Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.