BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies

• URL: http://arxiv.org/abs/2506.00328v3
• Date: Wed, 11 Jun 2025 06:03:44 GMT
• Title: BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies
• Authors: Kourosh Shahnazari, Seyed Moein Ayyoubzadeh, Mohammadali Keshtparvar,
• Abstract summary: BASIL (Best-Action Symbolic Interpretable Learning) is a systematic approach for generating symbolic, rule-based policies.<n>This article introduces a new interpretable policy synthesis method that combines symbolic expressiveness, evolutionary diversity, and online learning.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The quest for interpretable reinforcement learning is a grand challenge for the deployment of autonomous decision-making systems in safety-critical applications. Modern deep reinforcement learning approaches, while powerful, tend to produce opaque policies that compromise verification, reduce transparency, and impede human oversight. To address this, we introduce BASIL (Best-Action Symbolic Interpretable Learning), a systematic approach for generating symbolic, rule-based policies via online evolutionary search with quality-diversity (QD) optimization. BASIL represents policies as ordered lists of symbolic predicates over state variables, ensuring full interpretability and tractable policy complexity. By using a QD archive, the methodology in the proposed study encourages behavioral and structural diversity between top-performing solutions, while a complexity-aware fitness encourages the synthesis of compact representations. The evolutionary system supports the use of exact constraints for rule count and system adaptability for balancing transparency with expressiveness. Empirical comparisons with three benchmark tasks CartPole-v1, MountainCar-v0, and Acrobot-v1 show that BASIL consistently synthesizes interpretable controllers with compact representations comparable to deep reinforcement learning baselines. Herein, this article introduces a new interpretable policy synthesis method that combines symbolic expressiveness, evolutionary diversity, and online learning through a unifying framework.

Related papers

• Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why [50.191655141020505]
  This survey provides a comparative analysis of feature-based and GAN-based approaches to learning from demonstrations.<n>We argue that the dichotomy between feature-based and GAN-based methods is increasingly nuanced.
  arXiv  Detail & Related papers  (2025-07-08T11:45:51Z)
• QA-LIGN: Aligning LLMs through Constitutionally Decomposed QA [49.9801383018588]
  We introduce QA-LIGN, an automatic symbolic reward decomposition approach.<n>Instead of training a black-box reward model that outputs a monolithic score, QA-LIGN formulates principle-specific evaluation questions.<n>Experiments aligning an uncensored large language model with a set of constitutional principles demonstrate that QA-LIGN offers greater transparency and adaptability.
  arXiv  Detail & Related papers  (2025-06-09T18:24:57Z)
• From Explainability to Interpretability: Interpretable Policies in Reinforcement Learning Via Model Explanation [2.08099858257632]
  We propose a novel model-agnostic approach to transform complex deep RL policies into transparent representations.<n>We evaluate our approach with three existing deep RL algorithms and validate its performance in two classic control environments.
  arXiv  Detail & Related papers  (2025-01-16T22:11:03Z)
• Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization [9.035959289139102]
  We introduce SYMPOL, a novel method for SYMbolic tree-based on-policy RL.<n>SYMPOL employs a tree-based model integrated with a policy gradient method, enabling the agent to learn and adapt its actions.<n>We evaluate SYMPOL on a set of benchmark RL tasks, demonstrating its superiority over alternative tree-based RL approaches.
  arXiv  Detail & Related papers  (2024-08-16T14:04:40Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search [63.3745291252038]
  We propose DiffSES, a novel symbolic learning approach that discovers discrete symbolic policies. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more scalable than state-of-the-art symbolic RL methods.
  arXiv  Detail & Related papers  (2022-12-30T17:50:54Z)
• Bayesian Soft Actor-Critic: A Directed Acyclic Strategy Graph Based Deep Reinforcement Learning [1.8220718426493654]
  This paper proposes a novel directed acyclic strategy graph decomposition approach based on Bayesian chaining. We integrate this approach into the state-of-the-art DRL method -- soft actor-critic (SAC) We build the corresponding Bayesian soft actor-critic (BSAC) model by organizing several sub-policies as a joint policy.
  arXiv  Detail & Related papers  (2022-08-11T20:36:23Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)
• Verified Probabilistic Policies for Deep Reinforcement Learning [6.85316573653194]
  We tackle the problem of verifying probabilistic policies for deep reinforcement learning. We propose an abstraction approach, based on interval Markov decision processes, that yields guarantees on a policy's execution. We present techniques to build and solve these models using abstract interpretation, mixed-integer linear programming, entropy-based refinement and probabilistic model checking.
  arXiv  Detail & Related papers  (2022-01-10T23:55:04Z)
• Composable Learning with Sparse Kernel Representations [110.19179439773578]
  We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment.
  arXiv  Detail & Related papers  (2021-03-26T13:58:23Z)
• Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Video [128.08590291947544]
  Temporally language grounding in untrimmed videos is a newly-raised task in video understanding. Inspired by human's coarse-to-fine decision-making paradigm, we formulate a novel Tree-Structured Policy based Progressive Reinforcement Learning framework.
  arXiv  Detail & Related papers  (2020-01-18T15:08:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.