Explainable Reinforcement Learning via Physics-Aware Policy Distillation

• URL: http://arxiv.org/abs/2607.24672v1
• Date: Mon, 27 Jul 2026 17:14:42 GMT
• Title: Explainable Reinforcement Learning via Physics-Aware Policy Distillation
• Abstract summary: In safety-critical sectors such as robotics and automotive engineering, the deployment of Deep Reinforcement Learning (DRL) is often hindered by the black-box nature of deep neural networks.<n>This paper presents an experimental study aimed at making high-performance continuous control DRL systems interpretable.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In safety-critical sectors such as robotics and automotive engineering, the deployment of Deep Reinforcement Learning (DRL) is often hindered by the black-box nature of deep neural networks. This lack of transparency poses significant challenges for regulatory compliance and human-agent trust. This paper presents an experimental study aimed at making high-performance continuous control DRL systems interpretable. A policy distillation framework is implemented using the classic Inverted Pendulum benchmark. A high-performance Twin Delayed DDPG (TD3) agent serves as an opaque, continuous teacher model, whose policy is distilled into an interpretable student surrogate based on a shallow Decision Tree. By leveraging a custom physics-aware feature and "Noisy Oracle Rollouts" for dataset generation, the distillation process achieves performance equivalent to the expert teacher. Furthermore, comparative control theory analysis reveals a fundamental trade-off: transitioning from continuous to discrete rule-based control induces high-frequency Bang-Bang actuation and a stable bimodal limit cycle. Simulation results indicate that Bounded-Input Bounded-Output (BIBO) stability is maintained while providing both global and local interpretability for safe autonomous systems.

Related papers

• ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning [74.19730275669227]
  ARISE-RL is a novel full-cycle self-evolution framework for training open-ended agents.<n>We present Reward-Gated Self-Evolution Distillation (RG-SED), which distills a memory-augmented variant of the same policy back into itself.<n>We also present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning.
  arXiv  Detail & Related papers  (2026-09-01T10:54:13Z)
• Understanding and Stabilizing Deep Q-Learning via Controlled Bootstrapping and Regulated Value Dynamics [2.358468106592515]
  We provide a systematic analysis of instability in deep Q-learning from three complementary perspectives.<n>We identify a reward-triggered self-reinforcing trap and characteristic parameter spike dynamics.<n>Experiments on Atari-100K and Procgen demonstrate competitive performance and improved training stability.
  arXiv  Detail & Related papers  (2026-08-17T06:55:41Z)
• Unified Planning-Learning Framework for Robust UUV Navigation Under Partial Observability [1.8047694351309207]
  This paper presents an observation-only autonomy framework for Unmanned Underwater Vehicles (UUVs) navigation in dynamic underwater environments.<n>The proposed pipeline constructs occupancy maps solely from onboard sonar and depth image observations.<n>The framework integrates persistent occupancy mapping, global clearance-aware planning, and risk-aware local control.
  arXiv  Detail & Related papers  (2026-08-05T19:37:22Z)
• Purified OPSD: On-Policy Self-Distillation Without Losing How to Think [52.879387744920415]
  We find that on-policy self-distillation (OPSD) consistently fails on long chain-of-thought (long-CoT) reasoning models.<n>We propose a two-step solution: first, we construct a reference-only teacher to isolate the non-transferable component of the supervision signal.<n>Second, we use pointwise mutual information (PMI) as the mechanism to transform this residual into a well-formed PMI target distribution.
  arXiv  Detail & Related papers  (2026-07-02T14:33:07Z)
• Self-Distilled RLVR [57.37526213765131]
  We show that learning signals solely derived from the privileged teacher result in severe information leakage and unstable long-term training.<n>We propose textbfRLSD (textbfRLVR with textbfSelf-textbfDistillation)<n>This enables RLSD to simultaneously harness the strengths of both RLVR and OPSD, achieving a higher convergence ceiling and superior training stability.
  arXiv  Detail & Related papers  (2026-04-03T15:50:07Z)
• VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillation [14.33516456804297]
  On-Policy VLA Distillation (VLA-OPD) is a framework bridging the efficiency of online Reinforcement Learning (RL) with the robustness of offline Supervised Fine-Tuning (SFT)
  arXiv  Detail & Related papers  (2026-03-27T17:59:33Z)
• Adaptive recurrent flow map operator learning for reaction diffusion dynamics [0.9137554315375919]
  We develop an operator learner with adaptive recurrent training (DDOL-ART) using a robust recurrent strategy with lightweight validation milestones.<n>DDOL-ART learns one-step operators that remain stable under long rollouts and generalize zero-shot to strong shifts.<n>It is several-fold faster than a physics-based numerical-loss operator learner (NLOL) under matched settings.
  arXiv  Detail & Related papers  (2026-02-10T07:33:13Z)
• COVLM-RL: Critical Object-Oriented Reasoning for Autonomous Driving Using VLM-Guided Reinforcement Learning [55.83415345423854]
  We propose COVLM-RL, a novel end-to-end driving framework that integrates Critical Object-oriented (CO) reasoning with reinforcement learning (RL)<n>Experiments conducted in the CARLA simulator demonstrate that COVLM-RL significantly improves the success rate by 30% in trained driving environments.
  arXiv  Detail & Related papers  (2025-12-10T06:18:16Z)
• Reinforcement Learning for Self-Healing Material Systems [0.40334315349753025]
  This study frames the self-healing process as a Reinforcement Learning problem within a Markov Decision Process (MDP), enabling agents to autonomously derive optimal policies.<n>A comparative evaluation of discrete-action (Q-learning, DQN) and continuous-action (TD3) agents in a simulation environment revealed that RL controllers significantly outperform baselines, achieving near-complete material recovery.
  arXiv  Detail & Related papers  (2025-11-24T03:42:00Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• Control Synthesis of Cyber-Physical Systems for Real-Time Specifications through Causation-Guided Reinforcement Learning [3.608670495432032]
  Signal temporal logic (STL) has emerged as a powerful formalism of expressing real-time constraints.<n> reinforcement learning (RL) has become an important method for solving control synthesis problems in unknown environments.<n>We propose an online reward generation method guided by the online causation monitoring of STL.
  arXiv  Detail & Related papers  (2025-10-09T02:49:28Z)
• On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting [91.38734024438357]
  Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) are two prominent post-training paradigms for refining the capabilities and aligning the behavior of Large Language Models (LLMs)<n>Existing approaches that integrate SFT and RL often face the risk of disrupting established response patterns and inducing overfitting to expert data.<n>We propose CHORD, a framework for Controllable Harmonization of On- and Off-Policy Reinforcement Learning via Dynamic Weighting.
  arXiv  Detail & Related papers  (2025-08-15T11:20:03Z)
• Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior [51.60683890503293]
  We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. We show that pure supervised cloning can generate trajectories matching the per-time step distribution of arbitrary expert trajectories.
  arXiv  Detail & Related papers  (2023-07-27T04:27:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.