Enhancing RL Generalizability in Robotics through SHAP Analysis of Algorithms and Hyperparameters

• URL: http://arxiv.org/abs/2605.02867v1
• Date: Mon, 04 May 2026 17:41:04 GMT
• Title: Enhancing RL Generalizability in Robotics through SHAP Analysis of Algorithms and Hyperparameters
• Abstract summary: We propose an explainable framework that evaluates Reinforcement Learning performance across robotic environments.<n>We empirically analyze configuration impact patterns, and introduce SHAP-guided configuration selection to enhance generalization.
• Score: 4.978287867181505
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Despite significant advances in Reinforcement Learning (RL), model performance remains highly sensitive to algorithm and hyperparameter configurations, while generalization gaps across environments complicate real-world deployment. Although prior work has studied RL generalization, the relative contribution of specific configurations to the generalization gap has not been quantitatively decomposed and systematically leveraged for configuration selection. To address this limitation, we propose an explainable framework that evaluates RL performance across robotic environments using SHapley Additive exPlanations (SHAP) to quantify configuration impacts. We establish a theoretical foundation connecting Shapley values to generalizability, empirically analyze configuration impact patterns, and introduce SHAP-guided configuration selection to enhance generalization. Our results reveal distinct patterns across algorithms and hyperparameters, with consistent configuration impacts across diverse tasks and environments. By applying these insights to configuration selection, we achieve improved RL generalizability and provide actionable guidance for practitioners.

Related papers

• SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learning [4.819678320271634]
  Specification-guided reinforcement learning (RL) provides a principled framework for encoding complex, temporally extended tasks.<n>We introduce SpecRLBench, a benchmark designed to evaluate the capabilities of generalization-based RL methods.
  arXiv  Detail & Related papers  (2026-04-27T17:40:18Z)
• Poly-EPO: Training Exploratory Reasoning Models [62.82992914206963]
  We present a framework for post-training language models (LMs) that explicitly encourages optimistic exploration and promotes a synergy between exploration and exploitation.<n>We propose Polychromic Exploratory Policy Optimization (Poly-EPO), which instantiates this framework with an objective that explicitly synergizes exploration and exploitation.
  arXiv  Detail & Related papers  (2026-04-19T22:54:19Z)
• Benchmarking Reinforcement Learning via Stochastic Converse Optimality: Generating Systems with Known Optimal Policies [0.0]
  We introduce a rigorous benchmarking framework by extending converse optimality to discrete-time, control-affine, nonlinear systems with noise.<n>Our framework provides necessary and sufficient conditions, under which a prescribed value function and policy are optimal for constructed systems.<n>We validate it by automatically constructing diverse environments, demonstrating our framework's capacity for a controlled and comprehensive evaluation across algorithms.
  arXiv  Detail & Related papers  (2026-03-18T11:52:21Z)
• CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies [4.274782706616976]
  Reinforcement learning (RL) has achieved notable performance in high-dimensional sequential decision-making tasks.<n>We propose a biologically grounded RL architecture that incorporate large expansion, sparse connectivity, sparse activation, and dendritic-level modulation.<n> Experiments on noisy, high-dimensional RL benchmarks show that both the cerebellar architecture and dendritic modulation consistently improve sample efficiency, robustness, and generalization compared to conventional designs.
  arXiv  Detail & Related papers  (2026-02-17T05:25:09Z)
• Pimp My LLM: Leveraging Variability Modeling to Tune Inference Hyperparameters [0.3186130813218338]
  Large Language Models (LLMs) are being increasingly used across a wide range of tasks.<n>Their substantial computational demands raise concerns about the energy efficiency and sustainability of both training and inference.<n>Recent research has explored optimization techniques and analyzed how configuration choices influence energy consumption.
  arXiv  Detail & Related papers  (2026-02-06T16:18:22Z)
• TL-GRPO: Turn-Level RL for Reasoning-Guided Iterative Optimization [97.18886232580131]
  Large language models have demonstrated strong reasoning capabilities in complex tasks through tool integration.<n>We propose Turn-Level GRPO, a lightweight RL algorithm that performs turn-level group sampling for fine-grained optimization.
  arXiv  Detail & Related papers  (2026-01-23T06:21:33Z)
• Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds [63.47271262149291]
  We propose a unified framework for PAC-Bayesian norm-based generalization.<n>The key to our approach is a sensitivity matrix that quantifies the network outputs with respect to structured weight perturbations.<n>We derive a family of generalization bounds that recover several existing PAC-Bayesian results as special cases.
  arXiv  Detail & Related papers  (2026-01-13T00:42:22Z)
• Sample-Efficient Neurosymbolic Deep Reinforcement Learning [49.60927398960061]
  We propose a neuro-symbolic Deep RL approach that integrates background symbolic knowledge to improve sample efficiency.<n>Online reasoning is performed to guide the training process through two mechanisms.<n>We show improved performance over a state-of-the-art reward machine baseline.
  arXiv  Detail & Related papers  (2026-01-06T09:28:53Z)
• Hyperparameter Optimisation with Practical Interpretability and Explanation Methods in Probabilistic Curriculum Learning [2.5352713493505785]
  Probabilistic Curriculum Learning (PCL) is a curriculum learning strategy designed to improve RL performance by structuring the agent's learning process.<n>We provide an empirical analysis of hyperparameter interactions and their effects on the performance of a PCL algorithm within standard RL tasks.
  arXiv  Detail & Related papers  (2025-04-09T08:41:27Z)
• A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning [132.45959478064736]
  We propose a general framework that unifies model-based and model-free reinforcement learning. We propose a novel estimation function with decomposable structural properties for optimization-based exploration. Under our framework, a new sample-efficient algorithm namely OPtimization-based ExploRation with Approximation (OPERA) is proposed.
  arXiv  Detail & Related papers  (2022-09-30T17:59:16Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.