When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor State

• URL: http://arxiv.org/abs/2605.18580v1
• Date: Mon, 18 May 2026 15:58:34 GMT
• Title: When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor State
• Abstract summary: Outcome-only evaluation can certify economically unsafe agents.<n>In hotel pricing with hidden competitor state, a learner can achieve plausible revenue per available room while failing to preserve the rate discipline of a rule-based revenue-management policy.<n>We introduce discipline stability, a trace-based evaluation paradigm.
• Score: 0.39287497907611874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Outcome-only evaluation can certify economically unsafe agents: a policy can hit a business KPI while violating deployable behavioral discipline. In hotel pricing with hidden competitor state, a learner can achieve plausible revenue per available room while failing to preserve the rate discipline of a rule-based revenue-management competitor. We introduce discipline stability, a trace-based evaluation paradigm: define the benchmark behavior, restrict observations to the deployment regime, induce trace diagnostics from failure, separate mechanisms with ablations, and test transfer and deployment. Across a two-hotel benchmark and a compact hidden-budget bidding task, reward-only PPO variants miss trace alignment; revealing hidden state reduces label uncertainty; deterministic copy collapses uncertainty; and trace-prior or corrected history policies better preserve price or bid distributions. Pure behavior cloning is nearly enough for symmetric imitation, while Trace-Prior RL adds bounded adaptation under capacity asymmetry. The contribution is an evaluation and benchmark paradigm, not a new optimizer or a universal claim about MARL

Related papers

• RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges [48.831446371402414]
  RecurSE pairs a trainable judge with a synchronized policy-copy checker that audits the judge's reasoning against meta-rubrics to supply a scalar process reward.<n>We show that RecurSE achieves consistent generalization gains across held-out medical, pairwise, summarization, and professional benchmarks.
  arXiv  Detail & Related papers  (2026-08-25T08:35:04Z)
• Scaling, Lock-In, and Proxy Compliance: A Political Economy of Responsible AI [1.253312107729806]
  We develop a sequential political-economy model to address accountability for AI systems.<n>We show that an AI vendor chooses auditability and substantive mitigation, a deployer monitors after adoption while switching costs, and enforcement depends on verifiable evidence.<n>The results explain why documentation and standardized evaluations can coexist with persistent post-deployment harms.
  arXiv  Detail & Related papers  (2026-07-30T11:12:13Z)
• Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models [55.788110316999166]
  We compare reasoning models produced via supervised fine-tuning, RL-based post-training, and distillation against instruction-tuned baselines.<n>We observe that reasoning models often improve on reasoning benchmarks but exhibit alignment regressions.<n>These regressions are consistent with behavioral drift from the instruction-tuned baseline, measured by KL divergence.
  arXiv  Detail & Related papers  (2026-06-09T16:14:27Z)
• TAPO: Tool-Aware Policy Optimization via Credit Transfer for Multimodal Search Agents [54.08846865906602]
  We characterize credit misassignment as a systematic failure mode of GRPO in tool-augmented multimodal search agents.<n>We propose Tool-Aware Policy Optimization (TAPO), which exploits the parameter-determinism property of information-acquisition tools.
  arXiv  Detail & Related papers  (2026-06-04T07:15:43Z)
• Calibrating Conservatism for Scalable Oversight [9.553819152637493]
  We introduce Calibrated Collective Oversight (CCO), which aggregates diverse auxiliary scoring functions into a penalty measuring deviation from a conservative baseline.<n>CCO calibrates this conservatism online using Conformal Decision Theory, ensuring that undesirable outcomes remain below a user-specified target threshold.<n>On a modified version of SWE-bench, weaker overseers successfully constrain an adversarially misaligned stronger agent; on MACHIAVELLI, CCO substantially reduces ethical violations while preserving reward.
  arXiv  Detail & Related papers  (2026-05-27T17:56:47Z)
• Proper Scoring Rules for Agentic Uncertainty Quantification [0.038379177968040606]
  Trajectory Proper Score (TPS) is a predictor-agnostic family of strictly proper trajectory-level scoring rules.<n>Trajectory ECE is resolution-blind, while scalarized Trajectory Brier elicits only the collapsed scalar, not the full trace.
  arXiv  Detail & Related papers  (2026-05-23T22:22:00Z)
• Decision Support for Marketplace Policies under Incomplete Evidence: From Replay to Launch Readiness [0.5013248430919224]
  We propose a support-aware decision-support system (DSS) that distinguishes promising from actionable evidence.<n>The framework integrates replay, support-aware off-policy evaluation (OPE), conservative lower-bound ranking, multi-sided guardrails, out-of-time validation, sensitivity analysis, and interference-aware validation design.<n>Applying the framework to iPinYou-style RTB logs, we identify a margin-gated floor policy as the leading candidate.
  arXiv  Detail & Related papers  (2026-05-13T00:26:50Z)
• Market-Alignment Risk in Pricing Agents: Trace Diagnostics and Trace-Prior RL under Hidden Competitor State [0.39287497907611874]
  We study failure in a two-hotel revenue-management simulator.<n>Hotel A trains an agent against a fixed rule-based revenue-management competitor.
  arXiv  Detail & Related papers  (2026-05-07T16:31:39Z)
• ValueBlindBench: Agreement-Gated Stress Testing of LLM-Judged Investment Rationales Before Returns Are Observable [1.8689252029357564]
  This paper introduces ValueBlindBench, a preregistered agreement-gated stress-test protocol.<n>ValueBlindBench decides when LLM-judged investment-rationale claims are publishable, qualified, or invalid.
  arXiv  Detail & Related papers  (2026-04-28T05:04:20Z)
• The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation [67.26315138466312]
  We propose a calibration-aware OPD framework, CaOPD, that estimates empirical confidence from model rollouts.<n>Our findings highlight that capability distillation does not imply confidence, and that confidence should be treated as an essential objective in post-training.
  arXiv  Detail & Related papers  (2026-04-18T04:43:40Z)
• Bayesian Conservative Policy Optimization (BCPO): A Novel Uncertainty-Calibrated Offline Reinforcement Learning with Credible Lower Bounds [1.2183405753834562]
  offline reinforcement learning (RL) aims to learn decision policies from a fixed batch of logged transitions.<n>We propose emphBayesian Conservative Policy Optimization (BCPO), a unified framework that converts uncertainty into emphprovably conservative policy improvement.<n>BCPO maintains a hierarchical Bayesian posterior over environment/value models, constructs a emphcredible lower bound (LCB) on action values, and performs policy updates under explicit KL regularization.
  arXiv  Detail & Related papers  (2026-03-06T01:46:02Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• LegalSim: Multi-Agent Simulation of Legal Systems for Discovering Procedural Exploits [0.0]
  We present LegalSim, a modular multi-agent simulation of adversarial legal proceedings that explores how AI systems can exploit procedural weaknesses in codified rules.<n>We compare four policies: PPO, a contextual bandit with an LLM, a direct LLM policy, and a hand-crafted; Instead of optimizing binary case outcomes, agents are trained and evaluated using effective win rate and a composite exploit score that combines opponent-cost inflation, calendar pressure, settlement pressure at low merit, and a rule-compliance margin.
  arXiv  Detail & Related papers  (2025-10-03T18:01:57Z)
• Hallucinated Adversarial Control for Conservative Offline Policy Evaluation [64.94009515033984]
  We study the problem of conservative off-policy evaluation (COPE) where given an offline dataset of environment interactions, we seek to obtain a (tight) lower bound on a policy's performance. We introduce HAMBO, which builds on an uncertainty-aware learned model of the transition dynamics. We prove that the resulting COPE estimates are valid lower bounds, and, under regularity conditions, show their convergence to the true expected return.
  arXiv  Detail & Related papers  (2023-03-02T08:57:35Z)
• Delving into Probabilistic Uncertainty for Unsupervised Domain Adaptive Person Re-Identification [54.174146346387204]
  We propose an approach named probabilistic uncertainty guided progressive label refinery (P$2$LR) for domain adaptive person re-identification. A quantitative criterion is established to measure the uncertainty of pseudo labels and facilitate the network training. Our method outperforms the baseline by 6.5% mAP on the Duke2Market task, while surpassing the state-of-the-art method by 2.5% mAP on the Market2MSMT task.
  arXiv  Detail & Related papers  (2021-12-28T07:40:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.