Split the Labor: Separating Evidence Interpretation from Decision Aggregation

• URL: http://arxiv.org/abs/2608.14509v1
• Date: Fri, 14 Aug 2026 17:24:55 GMT
• Title: Split the Labor: Separating Evidence Interpretation from Decision Aggregation
• Abstract summary: Systems that ask a language model to reach a conclusion from many sources usually conflate them into one prompt.<n>We propose a four-field evidence (hypothesis, reliability bucket, rationale, provenance) and show that fixing it determines both halves.<n>The fix is arithmetic rather than architectural, and applies to a class of rules beyond language models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Systems that ask a language model to reach a conclusion from many sources usually concatenate them into one prompt. This conflates two operations with different requirements. Interpreting a source rewards capacity and context. Combining interpretations rewards fixed arithmetic, comparability across instances, and the option to return nothing. Once separated, the design problem becomes the interface between them. We propose a four-field evidence tuple (hypothesis, reliability bucket, rationale, provenance) and show that fixing it determines both halves. The separation also reveals a failure mode in how such systems combine, which we call count-scale drift. Thresholding a sum of unnormalized weights is exactly posterior thresholding, but at an operating point that slides with the number of sources consulted. The slide grows with reader reliability. When source reliabilities differ, the vote rule and the posterior order instances differently, and no threshold reconciles them. Pooling calibrated log-likelihood ratios addresses both problems. The fix is arithmetic rather than architectural, and applies to a class of rules beyond language models: score-summing triage engines, diagnostic panels scored by counting positives, and additive multi-signal detectors. We then instantiate the principle twice on one longitudinal corpus, once after outcomes resolve and once before. The same partition helps in both, at different granularities: over reading in the first, over learning capacity in the second. There, a small sequence encoder on an easy auxiliary objective plus a tree ensemble carrying the censored survival loss reaches 0.921 AUPRC against 0.805 for a hand-crafted baseline. We separate what transfers from what must be re-estimated per domain, and state five predictions that would falsify the framework, three negative results, and which comparisons remain confounded.

Related papers

• Calibrating Interpretability Instruments Before Trusting Their Verdicts [1.0829694003408499]
  Causal claims about large language model (LLM) internals rest on measurements.<n>These measurements fail in specific, diagnosable ways that return a plausible number instead of an error.<n>This note documents six such failures from a causal interpretability program on refusal and moral representation.
  arXiv  Detail & Related papers  (2026-09-13T19:37:02Z)
• Grouping the Stochastic Machine: Precision, Not Capability, as the Frontier Metric for AI Systems [0.0]
  I argue that frontier language models are compared, marketed, and benchmarked on capability -- what their best or average output can achieve.<n>What now separates one system from another in practice is precision.<n>I define a grouping metric, specify a harness, and show how tracking a human-AI pair's grouping over time yields the compounding signal that Paper 1's field study requires.
  arXiv  Detail & Related papers  (2026-08-19T17:29:47Z)
• Tracing Provenance and Detecting Tampering with Complementary LLM Watermarks [68.64050157343334]
  Existing watermarks preserve provenance under editing, but this same robustness allows an adversary to alter critical content while retaining attribution.<n>We introduce an innovative watermark that jointly provides provenance and tamper evidence.<n>The signals share the same mechanism but use independent keys and different seeding windows over normalized text, making one resilient to edits and the other sensitive to reader-visible changes.
  arXiv  Detail & Related papers  (2026-08-13T01:55:16Z)
• What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model [0.0]
  A class of methods probes a language model by feeding it its own output.<n>We ask what such a probe measures, in a construction chosen to make the question sharp.<n>The answer is that it measures two different things at once, in readings that look alike.
  arXiv  Detail & Related papers  (2026-08-11T14:40:35Z)
• Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk [0.0]
  Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters.<n>A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale.<n>We build such a system end to end and test whether the second half of the promise holds.
  arXiv  Detail & Related papers  (2026-08-08T13:22:14Z)
• When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering [0.8594140167290097]
  A model's agreement across perturbed inputs is used as a label-free reliability signal.<n>Rerendering beats resampling on both accuracy and reliability.<n>Fine-tuning on the model's own cross-render consensus inverts.
  arXiv  Detail & Related papers  (2026-08-06T07:08:05Z)
• The Equilibrium Is the Initialization: Lazy Identity Collapse in Physics-Structured Deep Equilibrium Reasoning [0.0]
  Deep equilibrium models promise input-adaptive implicit computation.<n>We report a cautionary study of a port-Hamiltonian DEQ with learned reasoning tasks.<n>We show that the standard zeroing ablation is confounded and gives wildly seed-dependent answers.
  arXiv  Detail & Related papers  (2026-07-13T05:45:35Z)
• Two Confounds in Cross-Model Value Comparison: Response Determinism and the Access Harness [0.0]
  Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values.<n>We introduce a separation protocol -- no-rule value dilemmas with counterbalanced, repeated forced-choice measurement and a determinism index.<n>We find determinism varies substantially (0.66-0.95 among engaging models)
  arXiv  Detail & Related papers  (2026-07-11T08:26:48Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Pando: Do Interpretability Methods Work When Models Won't Explain Themselves? [53.07826484214082]
  We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis.<n>Pando predicts held-out model decisions from $10$ labeled query-response pairs.<n>When explanations are faithful, black-box elicitation matches or exceeds all white-box methods.
  arXiv  Detail & Related papers  (2026-04-13T06:42:24Z)
• Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching [66.39914384073145]
  We propose a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates.<n>We find that step-level recombination is most beneficial on harder problems.<n>Our training-free framework improves average accuracy by up to 2 across six math and coding tasks.
  arXiv  Detail & Related papers  (2026-02-26T11:08:39Z)
• Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression [102.24287051757469]
  We study self-supervised covariance estimation in deep heteroscedastic regression.<n>We derive an upper bound on the 2-Wasserstein distance between normal distributions.<n>Experiments over a wide range of synthetic and real datasets demonstrate that the proposed 2-Wasserstein bound coupled with pseudo label annotations results in a computationally cheaper yet accurate deep heteroscedastic regression.
  arXiv  Detail & Related papers  (2025-02-14T22:37:11Z)
• Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization [73.04187954213471]
  We introduce a unified learning approach to simultaneously modeling the coarse- and fine-grained retrieval. The proposed method has achieved +4.03%, +3.38%, and +2.40% Recall@50 accuracy over a strong baseline.
  arXiv  Detail & Related papers  (2022-11-14T14:25:40Z)
• Modeling Voting for System Combination in Machine Translation [92.09572642019145]
  We propose an approach to modeling voting for system combination in machine translation. Our approach combines the advantages of statistical and neural methods since it can not only analyze the relations between hypotheses but also allow for end-to-end training.
  arXiv  Detail & Related papers  (2020-07-14T09:59:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.