Signed Evidence Flow: Conflict-Aware and Stability-Calibrated Data Analysis

• URL: http://arxiv.org/abs/2606.21875v1
• Date: Sat, 20 Jun 2026 04:29:07 GMT
• Title: Signed Evidence Flow: Conflict-Aware and Stability-Calibrated Data Analysis
• Abstract summary: We propose Signed Evidence Flow (SEF), which combines a fitted prediction rule with signed feature attributions to measure support, opposition, conflict, and stability.<n>We prove that confidence determines conflict exactly when it also determines total evidence mass, derive the remaining conditional variance, and state when conflict can improve loss prediction beyond confidence.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern data analysis usually gives a prediction without showing whether the evidence behind it is clear, conflicting, or stable. Two cases can have the same fitted confidence even when one has mostly agreeing evidence and the other has strong support and strong opposition. We propose Signed Evidence Flow (SEF), which combines a fitted prediction rule with signed feature attributions to measure support, opposition, conflict, and perturbation stability. We prove that confidence determines conflict exactly when it also determines total evidence mass, derive the remaining conditional variance, and state when conflict can improve loss prediction beyond confidence and other audit variables. We also connect conflict to geometric decision fragility. Across healthcare, Covertype, black-box, finance, and ten external data sets, conflict sometimes separates risk among predictions that already appear confident. Cross-fitted tests show added error-ranking information beyond confidence and attribution entropy on several data sets, including two large finance tasks. The direction is not universal: in some tasks, lowconflict cases are riskier. We therefore introduce ScopeGate, a held-out permutation diagnostic that checks the direction before SEF is used for review triage. SEF is consequently an audit tool rather than a universal risk score: it describes evidence structure, while an independent calibration sample determines whether that structure is useful in the target population.

Related papers

• When Confidence Signals Disagree: Local and Global Confidence in Autoregressive Language Models [0.0]
  We investigate whether different confidence readouts are empirically interchangeable in an autoregressive language model.<n>Across MMLU and ARC Challenge, the two signals are weakly correlated and differ substantially in their association with correctness.<n>Larger local--global confidence gaps are associated with higher answer entropy, more distinct sampled answers, and lower modal-answer concentration.
  arXiv  Detail & Related papers  (2026-09-15T10:08:48Z)
• FinAbstain: Uncertainty-Calibrated Multimodal RAG for Selective Financial Forecasting [8.891247829087936]
  FinAbstain is a framework for uncertainty-calibrated multimodal retrieval-augmented generation with selective prediction.<n>A point-in-time retriever admits only information public at the forecast timestamp and supplies modality-specific evidence to fundamental, news, technical, risk, and verification agents.<n>A controller predicts bullish, bearish, or neutral outcomes only when uncertainty is below a validated threshold.
  arXiv  Detail & Related papers  (2026-07-27T07:05:22Z)
• When Does Quality-Aware Multimodal Fusion Matter? A Leakage-Safe Diagnostic for Decision-Level Dependence [3.899171236602393]
  We propose a simple diagnostic to test whether reliability information is used during inference.<n>Experiments on stress recognition and CMU-MOSEI for sentiment analysis show that permuting reliability scores leaves performance unchanged.
  arXiv  Detail & Related papers  (2026-06-25T00:14:22Z)
• ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidence [48.938290419832335]
  We introduce ConflictScore, a novel metric that quantifies how well a model's response acknowledges conflicting evidence in its grounding documents.<n>Our framework decomposes responses into atomic claims, labels each claim against each grounding document, and then aggregates these labels into two complementary measures: ConflictScore-Count (CS-C), the proportion of claims exhibiting conflicts, and ConflictScore-Ratio (CS-R), the balance between supporting and contradicting evidence.
  arXiv  Detail & Related papers  (2026-06-24T23:00:09Z)
• Confident but Conflicted: Internal Uncertainty and Cognitive Dissonance Resolution in LLMs [6.968805961712596]
  Large language models (LLMs) frequently encounter inputs that disagree with their prior outputs.<n>Dissonance can be resolved through persuasion, backfire, or immunity.<n>Trust Elasticity (TE) is an econometrics-inspired measure of how readily a model is persuaded toward conflicting evidence.
  arXiv  Detail & Related papers  (2026-06-21T18:39:24Z)
• When Evidence Conflicts: Uncertainty and Order Effects in Retrieval-Augmented Biomedical Question Answering [2.6393907656700706]
  Using HealthContradict, we evaluate six open-weight large language models (LLMs) under five controlled evidence conditions.<n>Results show that conflicting biomedical evidence is both an uncertainty and robustness problem.
  arXiv  Detail & Related papers  (2026-05-13T21:02:24Z)
• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning [50.22167852149165]
  We propose EviDep, an evidential learning framework that jointly quantifies depression severity.<n>EviDep enforces strict information integrity to guarantee robust evidence synthesis.<n>It achieves state-of-the-art predictive accuracy and superior uncertainty calibration, delivering a robust fail-safe mechanism for trustworthy clinical screening.
  arXiv  Detail & Related papers  (2026-04-17T13:27:11Z)
• ERA: Evidence-based Reliability Alignment for Honest Retrieval-Augmented Generation [19.616905640490252]
  We propose a new framework called ERA (Evidence-based Reliability Alignment) to enhance abstention behavior in RAG systems.<n>Our approach significantly outperforms baselines, optimizing the trade-off between answer coverage and abstention with superior calibration.
  arXiv  Detail & Related papers  (2026-02-24T01:45:51Z)
• All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection [67.89888669159899]
  RFC Bench is a benchmark for evaluating large language models on financial misinformation under realistic news.<n>The benchmark defines two complementary tasks: reference free misinformation detection and comparison based diagnosis.
  arXiv  Detail & Related papers  (2026-01-07T18:18:28Z)
• Proximity-Based Evidence Retrieval for Uncertainty-Aware Neural Networks [6.9681910774977815]
  This work proposes an evidence-retrieval mechanism for uncertainty-aware decision-making.<n>For each test instance, exemplars are retrieved in an embedding space; their predictive distributions are fused via Dempster-Shafer theory.<n>Because the supporting evidences are explicit, decisions are transparent and auditable.
  arXiv  Detail & Related papers  (2025-09-11T13:12:22Z)
• Probabilistic Modeling of Disparity Uncertainty for Robust and Efficient Stereo Matching [61.73532883992135]
  We propose a new uncertainty-aware stereo matching framework.<n>We adopt Bayes risk as the measurement of uncertainty and use it to separately estimate data and model uncertainty.
  arXiv  Detail & Related papers  (2024-12-24T23:28:20Z)
• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regression [32.727673706238086]
  We propose a way of delving into the unexpected vulnerability in adversarially trained networks from a causal perspective. By deploying it, we estimate the causal relation of adversarial prediction under an unbiased environment. We demonstrate that the estimated causal features are highly related to the correct prediction for adversarial robustness.
  arXiv  Detail & Related papers  (2023-03-02T08:18:22Z)
• The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions [93.62888099134028]
  We find that the performance of state-of-the-art models on Natural Language Inference (NLI) and Reading (RC) analysis/stress sets can be highly unstable. This raises three questions: (1) How will the instability affect the reliability of the conclusions drawn based on these analysis sets? We give both theoretical explanations and empirical evidence regarding the source of the instability.
  arXiv  Detail & Related papers  (2020-04-28T15:41:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.