Groupoid-Based Internal State Representations for Reinforcement Learning with Local Symmetries

• URL: http://arxiv.org/abs/2609.13035v1
• Date: Fri, 11 Sep 2026 16:32:58 GMT
• Title: Groupoid-Based Internal State Representations for Reinforcement Learning with Local Symmetries
• Abstract summary: We propose a reinforcement learning framework using groupoids to capture local, state-dependent symmetries and support the dy- namic discovery of equivalence structures.<n> Empirical results demonstrate that the proposed groupoid-based approach improves sample efficiency and convergence in dense and large-scale environments exhibiting strong partial symmetries.
• Score: 1.1470070927586018
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Symmetries play a central role in reducing the complexity of reinforcement learning problems, yet most existing approaches rely on fixed group actions or predefined state abstractions. Classical reinforcement learning algorithms typically assume a globally structured Markov decision process with uniformly applicable actions and transitions, an assumption that limits their ability to exploit modularity and local, context-dependent regularities present in many realistic environments. We propose a reinforcement learning framework using groupoids to capture local, state-dependent symmetries and support the dy- namic discovery of equivalence structures during interaction. The agent maintains orbit representatives together with transporters that map raw states to canonical forms, enabling learning and decision-making to be performed in a symmetry-reduced space while preserving local distinctions. Empirical results demonstrate that the proposed groupoid-based approach improves sample efficiency and convergence in dense and large-scale environments exhibiting strong partial symmetries, yielding substantial performance gains over standard Q-learning. These findings show that dynamically exploiting local symmetry provides a practical and mathematically principled route to scalable and generalisable reinforcement learning.

Related papers

• Structured Representation Learning with Locally Linear Embeddings and Adaptive Feature Fusion [12.63651664394193]
  We propose a novel reinforcement learning framework that encourages the disentanglement of dynamics-specific and reward-specific features.<n>We show that our method improves learning efficiency and overall performance compared to conventional RL approaches.
  arXiv  Detail & Related papers  (2026-06-16T20:28:39Z)
• Equivariant Latent Alignment via Flow Matching under Group Symmetries [18.655367238679496]
  equivariant representation learning has emerged as a powerful framework for constructing latent spaces where analytically known group transformations could act directly.<n>Existing approaches often suffer from latent misalignment, a discrepancy between the intended group action and the actually required transformations in the latent space.<n>We propose Residual Latent Flow, a flow-based framework that corrects the misaligned latents.
  arXiv  Detail & Related papers  (2026-05-29T00:57:13Z)
• Anisotropic Modality Align [91.23979617826926]
  Training multimodal large language models has long been limited by the scarcity of high-quality paired multimodal data.<n>Recent studies show that the shared representation space of pretrained multimodal contrastive models can serve as a bridge, enabling models to perform multimodal training with unimodal data.<n>The core obstacle lies in the persistent Modality Gap in the shared space.
  arXiv  Detail & Related papers  (2026-05-08T14:53:24Z)
• Anticipatory Reinforcement Learning: From Generative Path-Laws to Distributional Value Functions [0.0]
  This paper introduces Anticipatory Reinforcement Learning (ARL), a novel framework designed to bridge the gap between non-Markovian decision processes and classical reinforcement learning architectures.<n>In environments characterised by jump-diffusions and structural breaks, traditional state-based methods often fail to capture the essential path-dependent geometry required for accurate foresight.
  arXiv  Detail & Related papers  (2026-04-06T13:15:44Z)
• Learning in Markov Decision Processes with Exogenous Dynamics [39.6376520918509]
  We study a structured class of MDPs characterized by state components whose transitions are independent of the agent's actions.<n>We show that exploiting this structure yields significantly improved learning guarantees.<n>We empirically validate our approach across classical toy settings and real-world-inspired environments.
  arXiv  Detail & Related papers  (2026-03-03T11:10:45Z)
• Invariance on Manifolds: Understanding Robust Visual Representations for Place Recognition [19.200074425090595]
  We propose a Second-Order Geometric Statistics framework that inherently captures geometric stability without training.<n>Our approach introduces a training-free framework built upon fixed, pre-trained backbones, achieving strong zero-shot generalization without parameter updates.
  arXiv  Detail & Related papers  (2026-01-31T18:12:29Z)
• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition [57.51370433303236]
  Chain-of-thought (CoT) reasoning enables large language models to break down complex problems into interpretable intermediate steps.<n>We introduce groundingS, a framework that formulates CoT reasoning as a Markov decision process (MDP) with latent state transitions.<n>We show improvements in reasoning accuracy, diversity, and exploration efficiency across benchmark reasoning tasks.
  arXiv  Detail & Related papers  (2025-07-10T21:32:18Z)
• Regularized Neural Ensemblers [55.15643209328513]
  In this study, we explore employing regularized neural networks as ensemble methods.<n>Motivated by the risk of learning low-diversity ensembles, we propose regularizing the ensembling model by randomly dropping base model predictions.<n>We demonstrate this approach provides lower bounds for the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Localisation in quasiperiodic chains: a theory based on convergence of local propagators [68.8204255655161]
  We present a theory of localisation in quasiperiodic chains with nearest-neighbour hoppings, based on the convergence of local propagators. Analysing the convergence of these continued fractions, localisation or its absence can be determined, yielding in turn the critical points and mobility edges. Results are exemplified by analysing the theory for three quasiperiodic models covering a range of behaviour.
  arXiv  Detail & Related papers  (2021-02-18T16:19:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.