Oracle-Guided Soft Shielding for Safe Move Prediction in Chess

• URL: http://arxiv.org/abs/2603.08506v1
• Date: Mon, 09 Mar 2026 15:40:01 GMT
• Title: Oracle-Guided Soft Shielding for Safe Move Prediction in Chess
• Abstract summary: Existing reinforcement learning approaches for environments such as chess require hundreds of thousands of episodes and substantial computational resources to converge.<n>We propose Oracle-Guided Soft Shielding (OGSS), a simple yet effective framework for safer decision-making.<n>Our results demonstrate that OGSS variants maintain a lower blunder rate even as the agent's exploration ratio is increased.
• Score: 2.731293010500297
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In high stakes environments, agents relying purely on imitation learning or reinforcement learning often struggle to avoid safety-critical errors during exploration. Existing reinforcement learning approaches for environments such as chess require hundreds of thousands of episodes and substantial computational resources to converge. Imitation learning, on the other hand, is more sample efficient but is brittle under distributional shift and lacks mechanisms for proactive risk avoidance. In this work, we propose Oracle-Guided Soft Shielding (OGSS), a simple yet effective framework for safer decision-making, enabling safe exploration by learning a probabilistic safety model from oracle feedback in an imitation learning setting. Focusing on the domain of chess, we train a model to predict strong moves based on past games, and separately learn a blunder prediction model from Stockfish evaluations to estimate the tactical risk of each move. During inference, the agent first generates a set of candidate moves and then uses the blunder model to determine high-risk options, and uses a utility function combining the predicted move likelihood from the policy model and the blunder probability to select actions that strike a balance between performance and safety. This enables the agent to explore and play competitively while significantly reducing the chance of tactical mistakes. Across hundreds of games against a strong chess engine, we compare our approach with other methods in the literature, such as action pruning, SafeDAgger, and uncertainty-based sampling. Our results demonstrate that OGSS variants maintain a lower blunder rate even as the agent's exploration ratio is increased by several folds, highlighting its ability to support broader exploration without compromising tactical soundness.

Related papers

• Self-Aware Active Learning Enables Continual Improvement in Autonomous Driving [12.669051115615176]
  Self-aware guided exploration (SAGE) is an active learning framework for post-training adaptation in autonomous driving.<n>Curiosity adaptively calibrates the intervention threshold for fear, allowing the agent to regulate risk in a context-dependent manner.<n>Fear is integrated into policy optimization and evaluation as a safety-oriented constraint to reduce performance regressions.
  arXiv  Detail & Related papers  (2026-08-30T13:01:32Z)
• Chess on Ice: Curling Tactical Decision-Making via Backward Induction and Deep Reinforcement Learning [10.932188857276605]
  We propose a reinforcement learning framework capable of quantitatively evaluating and comparing options in curling.<n>We employ the Deep Deterministic Policy Gradient actor-critic algorithm to exploit the finite-horizon structure of the game.<n>Our experiments show that effective curling strategies can be acquired in a fully self-supervised manner, without any human-annotated data.
  arXiv  Detail & Related papers  (2026-08-03T15:23:42Z)
• Fundamental Limits of Man-in-the-Middle Attack Detection in Model-Free Reinforcement Learning [2.9320341537404713]
  We consider the problem of learning-based man-in-the-middle (MITM) attacks in cyber-physical systems.<n>We refine the standard MDP attack model by allowing the reward function to depend on both the current and subsequent states.<n>We derive an optimal system-identification strategy for the adversary that minimizes detectable value deviations.
  arXiv  Detail & Related papers  (2026-03-29T09:18:00Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• Preliminary Investigation into Uncertainty-Aware Attack Stage Classification [81.28215542218724]
  This work addresses the problem of attack stage inference under uncertainty.<n>We propose a classification approach based on Evidential Deep Learning (EDL), which models predictive uncertainty by outputting parameters of a Dirichlet distribution over possible stages.<n>Preliminary experiments in a simulated environment demonstrate that the proposed model can accurately infer the stage of an attack with confidence.
  arXiv  Detail & Related papers  (2025-08-01T06:58:00Z)
• Exploiting Edge Features for Transferable Adversarial Attacks in Distributed Machine Learning [54.26807397329468]
  This work explores a previously overlooked vulnerability in distributed deep learning systems.<n>An adversary who intercepts the intermediate features transmitted between them can still pose a serious threat.<n>We propose an exploitation strategy specifically designed for distributed settings.
  arXiv  Detail & Related papers  (2025-07-09T20:09:00Z)
• Risk-Averse Reinforcement Learning with Itakura-Saito Loss [63.620958078179356]
  Risk-averse agents choose policies that minimize risk, occasionally sacrificing expected value.<n>We introduce a numerically stable and mathematically sound loss function based on the Itakura-Saito divergence for learning state-value and action-value functions.<n>In the experimental section, we explore multiple scenarios, some with known analytical solutions, and show that the considered loss function outperforms the alternatives.
  arXiv  Detail & Related papers  (2025-05-22T17:18:07Z)
• Sequential Manipulation Against Rank Aggregation: Theory and Algorithm [119.57122943187086]
  We leverage an online attack on the vulnerable data collection process. From the game-theoretic perspective, the confrontation scenario is formulated as a distributionally robust game. The proposed method manipulates the results of rank aggregation methods in a sequential manner.
  arXiv  Detail & Related papers  (2024-07-02T03:31:21Z)
• Automated Security Response through Online Learning with Adaptive Conjectures [13.33996350474556]
  We study automated security response for an IT infrastructure.<n>We formulate the interaction between an attacker and a defender as a partially observed, non-stationary game.
  arXiv  Detail & Related papers  (2024-02-19T20:06:15Z)
• Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis [63.532413807686524]
  This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL) We propose a new architecture that handles the trade-off between efficient progress and safety during exploration.
  arXiv  Detail & Related papers  (2023-12-18T16:09:43Z)
• Focus on Hiders: Exploring Hidden Threats for Enhancing Adversarial Training [20.1991376813843]
  We propose a generalized adversarial training algorithm called Hider-Focused Adversarial Training (HFAT) HFAT combines the optimization directions of standard adversarial training and prevention hiders. We demonstrate the effectiveness of our method based on extensive experiments.
  arXiv  Detail & Related papers  (2023-12-12T08:41:18Z)
• Learning Risk-Averse Equilibria in Multi-Agent Systems [13.25454171233235]
  In multi-agent systems, intelligent agents are tasked with making decisions that have optimal outcomes when the actions of the other agents are as expected. We introduce a new risk-averse solution concept that allows the learner to accommodate unexpected actions. We show that our population of agents that approximate a risk-averse equilibrium is particularly effective in the presence of unseen opposing populations.
  arXiv  Detail & Related papers  (2022-05-30T21:20:30Z)
• On Assessing The Safety of Reinforcement Learning algorithms Using Formal Methods [6.2822673562306655]
  Safety mechanisms such as adversarial training, adversarial detection, and robust learning are not always adapted to all disturbances in which the agent is deployed. It is therefore necessary to propose new solutions adapted to the learning challenges faced by the agent. We use reward shaping and a modified Q-learning algorithm as defense mechanisms to improve the agent's policy when facing adversarial perturbations.
  arXiv  Detail & Related papers  (2021-11-08T23:08:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.