Compositional Behavioral Semantics for State Abstraction in Reinforcement Learning

• URL: http://arxiv.org/abs/2606.25357v1
• Date: Wed, 24 Jun 2026 03:43:47 GMT
• Title: Compositional Behavioral Semantics for State Abstraction in Reinforcement Learning
• Abstract summary: We present a unified framework for defining and analyzing behavioral structures in reinforcement learning.<n>Our framework provides a compositional way to specify behavioral semantics based on local, one-step descriptions of system dynamics.<n>We show how to construct quantitative metrics from logical behavioral semantics with soundness guarantees.
• Score: 10.043594646353691
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State abstraction plays a key role in scaling reinforcement learning to complex but structured systems. In studying such systems, a wide range of behavioral structures have been studied in reinforcement learning, including value functions, invariants, bisimulation relations, and behavioral metrics. However, a general principle for determining what structures are provably preserved under state abstraction is still lacking. In this paper, we present a unified framework for defining and analyzing behavioral structures in reinforcement learning. Our framework provides a compositional way to specify behavioral semantics based on local, one-step descriptions of system dynamics. Using this framework, we establish results showing how behavioral structures can be safely transferred between abstract and concrete systems. We further show how to construct quantitative metrics from logical behavioral semantics with soundness guarantees. Together, these results provide a principled foundation for reasoning about behaviors under state abstraction in reinforcement learning and offer reusable definition and proof principles for a broad class of behavioral structures in reinforcement learning.

Related papers

• General Machine Learning: Theory for Learning Under Variable Regimes [2.538209532048866]
  We study learning under regime variation, where the learner, its memory state, and the evaluative conditions may evolve over time.<n>The paper develops a regime-varying framework centered on admissible transport, protected-core preservation, and evaluator-aware learning evolution.
  arXiv  Detail & Related papers  (2026-03-24T13:58:31Z)
• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• A Formal Descriptive Language for Learning Dynamics: A Five-Layer Structural Coordinate System [0.0]
  This paper proposes a multi-layer formal descriptive framework for learning dynamics.<n>Rather than offering a predictive or prescriptive model, the framework introduces a symbolic language composed of state variables, mappings, and layer-specific responsibilities.
  arXiv  Detail & Related papers  (2025-12-20T22:46:13Z)
• Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics [69.00587226225232]
  We introduce a state-aware transition framework that abstracts CoT trajectories into structured latent dynamics.<n>To characterize the global structure of reasoning, we model their progression as a Markov chain.<n>This abstraction supports a range of analyses, including semantic role identification, temporal pattern visualization, and consistency evaluation.
  arXiv  Detail & Related papers  (2025-08-29T18:53:31Z)
• Fuzzy, Symbolic, and Contextual: Enhancing LLM Instruction via Cognitive Scaffolding [3.553493344868413]
  We study how prompt-level inductive biases influence the cognitive behavior of large language models (LLMs) in instructional dialogue.<n>We introduce a symbolic scaffolding method paired with a short-term memory schema designed to promote adaptive, structured reasoning.<n>Preliminary results show that our full system consistently outperforms baseline variants.
  arXiv  Detail & Related papers  (2025-08-28T20:46:13Z)
• Understanding Learning Dynamics Through Structured Representations [1.7244210453129227]
  This paper investigates how internal structural choices shape the behavior of learning systems.<n>We analyze how these structures influence gradient flow, spectral sensitivity, and fixed-point behavior.<n>Rather than prescribing fixed templates, we emphasize principles of tractable design that can steer learning behavior in interpretable ways.
  arXiv  Detail & Related papers  (2025-08-04T07:15:57Z)
• Feature-Based vs. GAN-Based Learning from Demonstrations: When and Why [50.191655141020505]
  This survey provides a comparative analysis of feature-based and GAN-based approaches to learning from demonstrations.<n>We argue that the dichotomy between feature-based and GAN-based methods is increasingly nuanced.
  arXiv  Detail & Related papers  (2025-07-08T11:45:51Z)
• Sequential Representation Learning via Static-Dynamic Conditional Disentanglement [58.19137637859017]
  This paper explores self-supervised disentangled representation learning within sequential data, focusing on separating time-independent and time-varying factors in videos. We propose a new model that breaks the usual independence assumption between those factors by explicitly accounting for the causal relationship between the static/dynamic variables. Experiments show that the proposed approach outperforms previous complex state-of-the-art techniques in scenarios where the dynamics of a scene are influenced by its content.
  arXiv  Detail & Related papers  (2024-08-10T17:04:39Z)
• Inverse Decision Modeling: Learning Interpretable Representations of Behavior [72.80902932543474]
  We develop an expressive, unifying perspective on inverse decision modeling. We use this to formalize the inverse problem (as a descriptive model) We illustrate how this structure enables learning (interpretable) representations of (bounded) rationality.
  arXiv  Detail & Related papers  (2023-10-28T05:05:01Z)
• Generalized Reinforcement Learning: Experience Particles, Action Operator, Reinforcement Field, Memory Association, and Decision Concepts [2.398608007786179]
  This paper proposes a Bayesian-flavored generalized reinforcement learning framework. We first establish the notion of parametric action model to better cope with uncertainty and fluid action behaviors. We then introduce the notion of reinforcement field as a physics-inspired construct established through "polarized experience particles" maintained in the learning agent's working memory.
  arXiv  Detail & Related papers  (2022-08-09T15:05:15Z)
• On Neural Architecture Inductive Biases for Relational Tasks [76.18938462270503]
  We introduce a simple architecture based on similarity-distribution scores which we name Compositional Network generalization (CoRelNet) We find that simple architectural choices can outperform existing models in out-of-distribution generalizations.
  arXiv  Detail & Related papers  (2022-06-09T16:24:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.