A Structural Threshold in Decision Capacity Governs Collapse in Self-Play Reinforcement Learning

• URL: http://arxiv.org/abs/2605.16315v1
• Date: Mon, 04 May 2026 19:09:14 GMT
• Title: A Structural Threshold in Decision Capacity Governs Collapse in Self-Play Reinforcement Learning
• Abstract summary: We show that a threshold in decision capacity determines whether self-play reinforcement learning agents collapse under asymmetric rule perturbations.<n>Across poker variants, matrix games, a dice game, and multiple learning algorithms, eliminating all positive-reach contingent decisions causes rapid convergence to a deterministic exploitation attractor.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We show that a threshold in decision capacity determines whether self-play reinforcement learning agents collapse under asymmetric rule perturbations. Across poker variants, matrix games, a dice game, and multiple learning algorithms, eliminating all positive-reach contingent decisions causes rapid convergence to a deterministic exploitation attractor, a fixed point at near-maximal loss. Preserving even a single positive-reach contingent decision point prevents this collapse. A frozen baseline and fixed-opponent control confirm that the mechanism is co-adaptation under constraint, not the perturbation itself. The phenomenon is timing-invariant, fully reversible upon action restoration, and intensifies under function approximation. These results establish a sharp threshold at zero reach-weighted contingent action capacity, with severity scaling continuously via reach-weighted capacity in the tested domains.

Related papers

• Revelation Control [0.0]
  Revelation Control is the problem of choosing priced interventions that reveal hidden state only insofar as the revealed distinctions can change a consequential decision.<n>We develop this theory for learning systems, where states equivalent under declared current information can respond differently to future training and favor different actions.
  arXiv  Detail & Related papers  (2026-08-24T22:07:57Z)
• Plug-and-Play Reweighting for Resilient Collaborative Decision-Making in Connected Autonomous Driving [35.0382657479364]
  perceptual noise and adversarial attacks in collaborators can severely affect decision reliability.<n>We propose a novel Resilient Collaborative Decision-Making framework that consists of an attention-based encoder and an attention-based decoder.<n>Our approach consistently outperforms existing methods by up to 26% and achieves state-of-the-art resilient performance.
  arXiv  Detail & Related papers  (2026-07-10T23:36:31Z)
• Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning [63.24624171679711]
  Large language model (LLM)-based agents often make suboptimal tool-use decisions.<n>We propose TRUST, which incorporates uncertainty quantification into reward design as a repulsive force for maintaining uncertainty separation.
  arXiv  Detail & Related papers  (2026-06-05T07:08:34Z)
• Decision-calibrated prediction sets for robust power system operations [2.234083934481286]
  We introduce decisioncalibrated uncertainty sets to balance operating costs and reliability in power systems dominated by weather-dependent uncertainty.<n>These are defined in terms of the reliability of downstream decisions, rather than in terms of conditional coverage.<n>We show that decision-calibrated sets attain prescribed constraint-satisfaction targets within about three percentage points, whereas standard coverage-based calibration systematically exceeds these targets by more than eleven percentage points.
  arXiv  Detail & Related papers  (2026-06-01T11:12:09Z)
• AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning [13.721410595245894]
  State-dependent discounting is conceptually appealing, but naive deep actor--critic implementations can degenerate toward TD-error collapse.<n>We propose AdaGamma, a practical deep actor--critic method for state-dependent discounting that learns a state-dependent discount function together with a return-consistency objective.
  arXiv  Detail & Related papers  (2026-05-07T12:42:46Z)
• Neural Control: Adjoint Learning Through Equilibrium Constraints [7.41073008855977]
  We propose a boundary-control framework that computes trajectory-dependent, memory-efficient proxy gradients.<n>We show improved performance over gradient-free baselines such as SPSA and CEM.
  arXiv  Detail & Related papers  (2026-05-05T02:19:37Z)
• Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification [1.77898701462905]
  Evidential Deep Learning aims to address limitations by producing uncertainty estimates in a single pass.<n>This work introduces an alternative Dirichlet parameter estimation strategy by applying a method of moments estimator to ensembles of softmax outputs.<n>We show that the improved stability and predictive uncertainty behavior of these ensemble-derived Dirichlet estimates translate into stronger performance in downstream uncertainty-guided applications.
  arXiv  Detail & Related papers  (2026-04-07T16:28:20Z)
• Cognitive Friction: A Decision-Theoretic Framework for Bounded Deliberation in Tool-Using Agents [0.0]
  Unconstrained tool-using agents must decide which information source to query and act on.<n>We propose the Triadic Cognitive Architecture (TCA), a decision-theoretic framework that formalizes these failure modes via cognitive friction.<n>We validate TCA in two controlled environments designed to isolate stopping quality, action selection under congestion, and temporal urgency.
  arXiv  Detail & Related papers  (2026-03-31T17:30:25Z)
• Selection as Power: Constrained Reinforcement for Bounded Decision Authority [0.0]
  We introduce incentivized selection governance, where reinforcement updates are applied to scoring and reducer parameters under externally enforced sovereignty constraints.<n>We show that learning dynamics can coexist with structural diversity when sovereignty constraints are enforced at every update step.<n>These results demonstrate that learning dynamics can coexist with structural diversity when sovereignty constraints are enforced at every update step.
  arXiv  Detail & Related papers  (2026-03-02T16:02:34Z)
• Agentic Uncertainty Quantification [76.94013626702183]
  We propose a unified Dual-Process Agentic UQ (AUQ) framework that transforms verbalized uncertainty into active, bi-directional control signals.<n>Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, UAM), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, UAR), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary.
  arXiv  Detail & Related papers  (2026-01-22T07:16:26Z)
• Verifying Closed-Loop Contractivity of Learning-Based Controllers via Partitioning [52.23804865017831]
  We address the problem of verifying closed-loop contraction in nonlinear control systems whose controller and contraction metric are both parameterized by neural networks.<n>We derive a tractable and scalable sufficient condition for closed-loop contractivity that reduces to checking that the dominant eigenvalue of a symmetric Metzler matrix is nonpositive.
  arXiv  Detail & Related papers  (2025-12-01T23:06:56Z)
• Exploiting hidden structures in non-convex games for convergence to Nash equilibrium [62.88214569402201]
  A wide array of modern machine learning applications can be formulated as non-cooperative Nashlibria. We provide explicit convergence guarantees for both deterministic and deterministic environments.
  arXiv  Detail & Related papers  (2023-12-27T15:21:25Z)
• Anti-Exploration by Random Network Distillation [63.04360288089277]
  We show that a naive choice of conditioning for the Random Network Distillation (RND) is not discriminative enough to be used as an uncertainty estimator. We show that this limitation can be avoided with conditioning based on Feature-wise Linear Modulation (FiLM) We evaluate it on the D4RL benchmark, showing that it is capable of achieving performance comparable to ensemble-based methods and outperforming ensemble-free approaches by a wide margin.
  arXiv  Detail & Related papers  (2023-01-31T13:18:33Z)
• Model-Free Reinforcement Learning with the Decision-Estimation Coefficient [79.30248422988409]
  We consider the problem of interactive decision making, encompassing structured bandits and reinforcement learning with general function approximation. We use this approach to derive regret bounds for model-free reinforcement learning with value function approximation, and give structural results showing when it can and cannot help more generally.
  arXiv  Detail & Related papers  (2022-11-25T17:29:40Z)
• On the Complexity of Adversarial Decision Making [101.14158787665252]
  We show that the Decision-Estimation Coefficient is necessary and sufficient to obtain low regret for adversarial decision making. We provide new structural results that connect the Decision-Estimation Coefficient to variants of other well-known complexity measures.
  arXiv  Detail & Related papers  (2022-06-27T06:20:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.