Admission Without Answers: Label-Free Certification and Experience Learning for LLM-Based Optimization Modeling

• URL: http://arxiv.org/abs/2608.15565v3
• Date: Wed, 19 Aug 2026 05:13:37 GMT
• Title: Admission Without Answers: Label-Free Certification and Experience Learning for LLM-Based Optimization Modeling
• Abstract summary: AdmitOR is an admission gate built on calibrated external behavioral evidence.<n>It is the first label-free admission mechanism designed around an explicitly calibrated false-discovery target.
• Score: 8.467936416935986
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Experience-learning agents for optimization modeling improve by storing verified skills, but existing learners admit knowledge by checking against known answers, which real ticket streams do not provide. The natural label-free alternatives are unreliable: on a 300-problem label-blind stream, admitting every executable model poisons roughly one admission in four, while single-instance agreement accepts models that match at one value but differ elsewhere. We propose AdmitOR, an admission gate built on calibrated external behavioral evidence. Candidates from three model families, prompting strategies, and solver stacks are run on instances resampled from an extracted parameter domain; agreement across the resulting value-function traces is summarized by a cross-family clique, and a calibrated threshold returns accept, abstain, or escalate. The preregistered false-discovery criterion holds on calibration data but not on the wild stream. We report this negative result in full and trace most failures to benchmark texts that do not faithfully encode their labeled instances. Comparing four admission judges on one collection of logs inside a state-of-the-art skill learner, AdmitOR raises admission precision to 0.927, against 0.871 for majority vote and 0.726 for execution success, yielding 3.1x and 8.0x fewer poisoned admissions. Its library is the smallest and attains the highest macro accuracy across five public benchmarks, 58.4 against 54.8 for majority vote and 53.9 for the ground-truth-labeled library. The 3.5-point gain over majority vote is supported by a paired bootstrap and survives correction for a host-side anomaly. To our knowledge, AdmitOR is the first label-free admission mechanism designed around an explicitly calibrated false-discovery target. The transfer failure identifies a necessary condition for extending it to wild streams.

Related papers

• Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR [0.0]
  Reinforcement learning with rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text answer into a binary reward.<n>Prior work reports that one evaluation harness accepts only about 94% of its own ground truth answers, blaming parsing.<n>We supply the decomposition. We apply metamorphic testing to the verifier rather than the model, generating certified equivalent answer variants.<n>We measure rejection per answer category across four widely used verifiers over 307,420 verdicts.
  arXiv  Detail & Related papers  (2026-09-01T14:57:59Z)
• The C-index illusion: discrimination without calibration in published survival models [0.0]
  We reproduce three published survival-ML models across three structurally distinct domains.<n>We test five pre-registered hypotheses under a Holm-corrected family-wise error rate.
  arXiv  Detail & Related papers  (2026-07-21T19:13:54Z)
• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits [1.7545090618311434]
  Large language models translate natural-language descriptions of decision problems into solver-ready optimization models, but they fail silently.<n>This paper develops a theory of falsification-based verification for this setting.<n>Every numeric quantity in the description is a typed slot, and a candidate model is tested only through solver calls on slot-transformed instances.
  arXiv  Detail & Related papers  (2026-07-18T05:32:37Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• VERDI: Single-Call Confidence Estimation for Verification-Based LLM Judges via Decomposed Inference [0.0]
  VERDI (VERification-Decomposed Inference Inference) is a method that extracts confidence from a structured judge.<n> VERDI decomposes each evaluation into sub-checks and derives three structural signals.<n>On three public benchmarks, VERDI achieves AUROC 0.72-0.91 on GPT-4.1-mini and 0.66-0.80 on GPT-5.4-mini.
  arXiv  Detail & Related papers  (2026-05-11T23:39:19Z)
• Response Time Enhances Alignment with Heterogeneous Preferences [49.69696266152175]
  We show that augmenting preference datasets with a simple, secondary signal can restore identifiability of the population's average preference.<n>Our results bring promises and open up new opportunities for future data-collection pipelines.
  arXiv  Detail & Related papers  (2026-05-07T22:05:23Z)
• The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime [0.0]
  Most cited calibration result in deep learning -- post-temperature-scaling ECE of 0.012 on CIFAR-100 -- is below the statistical noise floor.<n>We prove this is not a failure of the experiment but a law: the minimax rate for estimating calibration error with model error rate epsilon is Theta((Lepsilon/m)2/3), and no estimator can beat it.
  arXiv  Detail & Related papers  (2026-04-14T16:48:24Z)
• CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution [52.691495954442985]
  CoVerRL is a framework where a single model alternates between generator and verifier roles, with each capability bootstrapping the other.<n> Experiments across Qwen and Llama model families demonstrate that CoVerRL outperforms label-free baselines by 4.7-5.9% on mathematical reasoning benchmarks.<n>Self-verification accuracy improves from around 55% to over 85%, confirming that both capabilities genuinely co-evolve.
  arXiv  Detail & Related papers  (2026-03-18T14:38:55Z)
• Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models [55.94503936470247]
  Large-scale AI evaluation increasingly relies on aggregating binary judgments from $K$ annotators, including judges.<n>Most classical methods assume annotators are conditionally independent given the true label $Yin0,1$, an assumption often violated by LLM judges.<n>We study label aggregation through a hierarchy of dependence-aware models based on Ising graphical models and latent factors.
  arXiv  Detail & Related papers  (2026-01-29T21:26:50Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning [47.21354101796544]
  This paper presents a statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the support of unlabeled instances for few-shot visual recognition. We rank the credibility of pseudo-labeled instances along the regularization path of their corresponding incidental parameters, and the most trustworthy pseudo-labeled examples are preserved as the augmented labeled instances.
  arXiv  Detail & Related papers  (2020-07-15T03:38:09Z)
• TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [80.38130122127882]
  TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE) In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled.
  arXiv  Detail & Related papers  (2020-04-30T15:07:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.