SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement

• URL: http://arxiv.org/abs/2607.18960v1
• Date: Tue, 21 Jul 2026 10:52:04 GMT
• Title: SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement
• Abstract summary: We present sys, a statistics-first gating architecture that treats procurement as a cost-aware routing problem.<n>Gate accepts a decision only when intervals are tight, sample sizes are adequate, and the axes agree.<n>We report honest negative diagnostics of the debate path: a con-side win rate of 0.80 and a 52% position-flip rate under advocate swapping.
• Score: 2.8129261352670003
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Procuring supervised fine-tuning (SFT) data forces a buyer to decide, before any downstream training, whether a candidate corpus is worth acquiring. We present \sys{}, a statistics-first gating architecture that treats procurement as a cost-aware routing problem over three intrinsic quality axes -- diversity, utility, and redundancy. Cheap blind measurements are summarised into per-axis estimates with confidence intervals; a gate accepts a decision only when intervals are tight, sample sizes are adequate, and the axes agree, otherwise it escalates the case to an adjudicative debate between a buy-advocate and a reject-advocate judge, resolved by a presiding verdict. On a controlled benchmark of 12 datasets ($2{\times}3{\times}2$ grid over the three axes) with 5 seeds, the gate reaches 0.90 accuracy and 0.83 $F_1$ at \$0.017 per unit, sitting between an always-verify baseline (0.75) and an oracle upper bound (0.98) while spending less than always-escalate (\$0.020). We further report honest negative diagnostics of the debate path: a con-side win rate of 0.80 ($p\approx3{\times}10^{-6}$) and a 52\% position-flip rate under advocate swapping expose negativity and positional biases that a naive LLM-judge would hide. We frame the injected-knob evaluation explicitly as a controlled synthetic benchmark for measurement fidelity and routing calibration, and delimit external validity as future work.

Related papers

• When Should a World Model Move? Loss-Conditioned State Execution [4.142275163201261]
  We introduce loss-conditioned state execution, a model-agnostic method that decides whether to execute a world model's fixed feasible proposal or retain the current state.<n>We formalize state movability as the existence of a loss-reducing feasible correction and distinguish it from the benefit of a particular proposal.<n>Experiments on public forecasting and action-conditioned dynamics benchmarks show supported updates and a trade-off between certification and coverage.
  arXiv  Detail & Related papers  (2026-09-14T16:12:03Z)
• Pricing the Risk of Runtime Compression: Anytime-Valid Admission and a Served-Output Law for Compressed Serving State [6.272878315342466]
  We show a union budget exhausts on every long request in a production serving stack (100% of requests)<n>We replace it with an anytime-valid, physically accounted ledger whose bound holds at every one of 352,333 admission calls on live traffic.<n>A priced bound is worth nothing on a request one has not seen, so the third link is the quantifier.
  arXiv  Detail & Related papers  (2026-08-16T15:37:44Z)
• Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk [0.0]
  Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters.<n>A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale.<n>We build such a system end to end and test whether the second half of the promise holds.
  arXiv  Detail & Related papers  (2026-08-08T13:22:14Z)
• Detecting and Discriminating Operator Misspecification in Hybrid PDE-Parameter Learning: a Reference-Free Instrument, with Discrimination Bounded In Sample [0.0]
  We build an instrument that reads from a single fit and with no oracle.<n>We report where the instrument is blind, a pre-registered negative where a neural estimator loses to Tikhonov-regularized inversion at recovery.
  arXiv  Detail & Related papers  (2026-08-04T21:20:44Z)
• Uncertainty-guided active learning for surrogate prediction of stream-finishing wear fields [37.82552353322716]
  We present an uncertainty-guided surrogate framework that predicts, directly from geometry, the three fields governing erosion.<n>The surrogate employs a deep ensemble whose disagreement estimates uncertainty, enabling an active-learning strategy that performs DEM simulations for the most uncertain orientations.
  arXiv  Detail & Related papers  (2026-08-01T11:12:39Z)
• One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence [41.99844472131922]
  A single human must audit $N$ LLM agents under a budget of $B ll N$ audits per round.<n>We model this as budgeted noisy inspection over a two-level Gaussian copula and locate the miscalibration threshold $*$ past which confidence-ranked auditing is emphworse than random.
  arXiv  Detail & Related papers  (2026-07-30T14:52:36Z)
• Trustworthy synthetic data for campaign decision support: strategy simulation fidelity and the PolicySynth framework [0.8757175705879833]
  Decision support systems increasingly run retention what-if analysis on synthetic customer populations, because privacy constraints preclude unrestricted use of real data.<n>We present three contributions: strategy simulation fidelity (SSF), a criterion measuring how often the synthetic population yields the same go/no-go campaign decision as the real population.<n>Policy Synth attains a mean SSF of 0.923 and 0.960, with seed-to-seed variance roughly ten times tighter than CTGAN on telecommunications and 2.5 times on banking.
  arXiv  Detail & Related papers  (2026-07-13T08:51:39Z)
• Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agents [45.148328075418156]
  We study a deployed multi-turn food-and-beverage ordering agent and measure how many genuine quality problems it catches.<n>Our blind-spot taxonomy shows the failure is structured, not random.<n>For production multi-turn agents, automated judging is a regression floor, not a substitute for human review.
  arXiv  Detail & Related papers  (2026-06-09T02:11:01Z)
• A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control [0.9668407688201359]
  We give a certificate for bounded, possibly non-monotone losses by treating the selected risk directly as a ratio.<n>Relative to the range-only Hoeffding-ratio construction this sharpens the acceptance-floor dependence from $1/pmin$ to $1/sqrtpmin$.<n>The certificate opens a $+22$ pp certified-acceptance frontier over Hoeffding--CRC.
  arXiv  Detail & Related papers  (2026-06-07T08:55:44Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• Multi-Dimensional Behavioral Evaluation of Agentic Stock Prediction Systems Using Large Language Model Judges with Closed-Loop Reinforcement Learning Feedback [1.2362187555287152]
  Forecast evaluation in finance has relied on aggregate accuracy metrics and predictive-accuracy tests built on point-forecast errors.<n>We propose a behavioral forecast-evaluation methodology that complements accuracy tests by assessing the intermediate decision process itself.
  arXiv  Detail & Related papers  (2026-05-07T06:31:34Z)
• Training-Free Probabilistic Time-Series Forecasting with Conformal Seasonal Pools [0.0]
  Conformal Seasonal Pools is a training-free probabilistic time-series forecaster.<n> CSP achieves all of this with no learned parameters and no training.
  arXiv  Detail & Related papers  (2026-05-05T14:16:35Z)
• Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity [0.3951796994513004]
  We introduce textbfIncompressible Knowledge Probes (IKPs), a benchmark of 1,400 factual questions spanning 7 tiers of obscurity.<n>We calibrate a log-linear mapping from IKP accuracy to parameter count on 89 open-weight models (135M-1,600B) spanning 19 vendors.<n>For Mixture-of-Experts models, total parameters predict knowledge far better than active parameters.
  arXiv  Detail & Related papers  (2026-04-27T15:46:23Z)
• Learning in Prophet Inequalities with Noisy Observations [38.27113852163736]
  We study the prophet inequality, a fundamental problem in online decision-making and optimal stopping.<n>We propose algorithms that integrate learning and decision-making via lower-confidence-bound thresholding.
  arXiv  Detail & Related papers  (2026-04-02T08:55:56Z)
• ZIP-RC: Optimizing Test-Time Compute via Zero-Overhead Joint Reward-Cost Prediction [57.799425838564]
  We present ZIP-RC, an adaptive inference method that equips models with zero-overhead inference-time predictions of reward and cost.<n> ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost.
  arXiv  Detail & Related papers  (2025-12-01T09:44:31Z)
• ORCA: Open-ended Response Correctness Assessment for Audio Question Answering [41.72231074041232]
  We present ORCA, a framework that models the variability in human judgments using Beta distributions to predict both expected correctness and uncertainty.<n>We collected 11,721 annotations across 3,580 question-answer pairs from 15 LALMs on two audio QA benchmarks, achieving inter-annotator agreement of 0.82 (Krippendorff's alpha)
  arXiv  Detail & Related papers  (2025-11-28T14:41:48Z)
• Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning [50.93804891554481]
  We introduce a novel estimator based on the log-sum-exponential (LSE) operator, which outperforms traditional inverse propensity score estimators.<n>Our LSE estimator demonstrates variance reduction and robustness under heavy-tailed conditions.<n>In the off-policy learning scenario, we establish bounds on the regret -- the performance gap between our LSE estimator and the optimal policy.
  arXiv  Detail & Related papers  (2025-06-07T17:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.