Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes

• URL: http://arxiv.org/abs/2607.27132v1
• Date: Wed, 29 Jul 2026 17:03:35 GMT
• Title: Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes
• Abstract summary: We describe a structured POMDP class in which the visible dynamics are RL and every realized visible transition applies a fixed permutation to a hidden mode.<n>We prove that the pair of current observation and stable class forms an exact finite Markov state.<n>Experiments recover an exact compression from raw states to quotient states and achieve perfect paired-order accuracy with three decision-time memory states.
• Score: 13.539613784615605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An agent acting under partial observability must retain a recursively updateable statistic of history that restores the Markov property, but the smallest such statistic is generally unknown. We characterize this minimal Markov sufficient statistic for holonomy-cover decision processes, a structured POMDP class in which the visible dynamics are Markov and every realized visible transition applies a fixed permutation to a hidden mode. In particular, we construct the stable quotient, the coarsest observation-wise abstraction preserving one-step rewards and quotient successors, and prove that the pair of the current observation and stable class forms an exact finite Markov state. When the current class is correctly initialized, exact class tracking requires exactly the minimal memory symbols, in the sense that under reachability and pairwise decision separation at a maximizing observation, no arbitrary finite-memory controller can use fewer. Under resettable diagnostics, nearest-prototype class inference has exponentially decaying error, and a calibrate-then-restart reduction transfers finite-MDP guarantees to the recovered state. The results enable \emph{Holonomy Memory Reinforcement Learning}. It represents memory by the current stable class, updates it through ordered edge transports, identifies local class coordinates when diagnostics are available, and applies a standard finite-MDP RL backbone after synchronization. Experiments recover an exact compression from raw states to quotient states and achieve perfect paired-order accuracy with three decision-time memory states, matching the quotient oracle and outperforming the non-oracle baselines.

Related papers

• Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning [22.455121713091245]
  Class unlearning aims to remove a model's ability to recognize designated forget classes while preserving performance on retain classes.<n>We study class relearning in a strictly source-free setting, asking whether a forget class can be recovered using only the unlearned model.
  arXiv  Detail & Related papers  (2026-09-02T02:45:56Z)
• Learning Latent Memory States from Longitudinal Athlete Monitoring Data [0.5729101515196012]
  We propose a new unit of analysis for longitudinal data: the Latent Memory Table.<n>It is that table, treated as a reusable statistical object on the same footing as a matrix of principal-component scores, a table of estimated random effects, or a table of predicted probabilities.
  arXiv  Detail & Related papers  (2026-08-06T17:14:54Z)
• Controller-Augmented Hidden Markov Models: A Computational Framework for Constrained Sequential Inference [0.0]
  We present Controller-Augmented Hidden Markov Models (CHMMs)<n>CHMMs is a framework that compiles each constraint into a finite-state controller tracking the minimal sufficient history.<n>We evaluate CHMMs against 6 alternative decoders on 3 real-world sequence-labeling tasks.
  arXiv  Detail & Related papers  (2026-06-11T19:29:09Z)
• Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Making [12.04727482907937]
  CML addresses the problem of tracking the noise of primal feedback multipliers under mini-batch conditions.<n>We decompose the multiplier into an effective pressure signal for primal descent and a finite-gain multiplier.
  arXiv  Detail & Related papers  (2026-06-05T09:35:22Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Relaxed Sparsest-Permutation Formulation for Causal Discovery at Scale [1.8160945635344523]
  We show that exact Cholesky factorization is unnecessary for structure recovery.<n>We introduce SCOPE, a sparse-Cholesky pipeline that provides a scalable implementation of the relaxed formulation.<n>Experiments on synthetic and real datasets demonstrate that SCOPE matches the MEC recovery accuracy of substantially slower baselines.
  arXiv  Detail & Related papers  (2026-05-07T01:20:29Z)
• Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration [76.08899010904652]
  CapCal is a training-free framework that mechanically decouples positional bias from ranking decisions.<n>It achieves superior performance among training-free methods while preserving single-pass efficiency.
  arXiv  Detail & Related papers  (2026-04-11T10:47:22Z)
• ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning [52.11294707895649]
  This paper explores the problem of class-agnostic anomaly detection (AD)<n>The objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data.<n> Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes.
  arXiv  Detail & Related papers  (2025-09-28T08:41:05Z)
• Post-Hoc Split-Point Self-Consistency Verification for Efficient, Unified Quantification of Aleatoric and Epistemic Uncertainty in Deep Learning [5.996056764788456]
  Uncertainty quantification (UQ) is vital for trustworthy deep learning, yet existing methods are either computationally intensive or provide only partial, task-specific estimates.<n>We propose a post-hoc single-forward-pass framework that jointly captures aleatoric and epistemic uncertainty without modifying or retraining pretrained models.<n>Our method applies emphSplit-Point Analysis (SPA) to decompose predictive residuals into upper and lower subsets, computing emphMean Absolute Residuals (MARs) on each side.
  arXiv  Detail & Related papers  (2025-09-16T17:16:01Z)
• Shift Before You Learn: Enabling Low-Rank Representations in Reinforcement Learning [56.87989363424]
  We show that a low-rank structure naturally emerges in the shifted successor measure.<n>We quantify the amount of shift needed for effective low-rank approximation and estimation.
  arXiv  Detail & Related papers  (2025-09-05T15:48:20Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning [49.94607673097326]
  We propose a highly adaptable framework, designated as SimPro, which does not rely on any predefined assumptions about the distribution of unlabeled data. Our framework, grounded in a probabilistic model, innovatively refines the expectation-maximization algorithm. Our method showcases consistent state-of-the-art performance across diverse benchmarks and data distribution scenarios.
  arXiv  Detail & Related papers  (2024-02-21T03:39:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.