Cerebellar-Inspired Residual Control for Fault Recovery: From Inference-Time Adaptation to Structural Consolidation

• URL: http://arxiv.org/abs/2602.07227v1
• Date: Fri, 06 Feb 2026 22:16:00 GMT
• Title: Cerebellar-Inspired Residual Control for Fault Recovery: From Inference-Time Adaptation to Structural Consolidation
• Abstract summary: We introduce an inference-time, cerebellar-inspired residual control framework that augments a frozen reinforcement learning policy with online corrective actions.<n>The framework instantiates core cerebellar principles, including high-dimensional pattern separation via fixed feature expansion.<n>Experiments on MuJoCo benchmarks show improvements of up to $+66%$ on textttHalfCheetah-v5 and $+53%$ on textttHumanoid-v5 under moderate faults.
• Score: 2.0421986354783437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic policies deployed in real-world environments often encounter post-training faults, where retraining, exploration, or system identification are impractical. We introduce an inference-time, cerebellar-inspired residual control framework that augments a frozen reinforcement learning policy with online corrective actions, enabling fault recovery without modifying base policy parameters. The framework instantiates core cerebellar principles, including high-dimensional pattern separation via fixed feature expansion, parallel microzone-style residual pathways, and local error-driven plasticity with excitatory and inhibitory eligibility traces operating at distinct time scales. These mechanisms enable fast, localized correction under post-training disturbances while avoiding destabilizing global policy updates. A conservative, performance-driven meta-adaptation regulates residual authority and plasticity, preserving nominal behavior and suppressing unnecessary intervention. Experiments on MuJoCo benchmarks under actuator, dynamic, and environmental perturbations show improvements of up to $+66\%$ on \texttt{HalfCheetah-v5} and $+53\%$ on \texttt{Humanoid-v5} under moderate faults, with graceful degradation under severe shifts and complementary robustness from consolidating persistent residual corrections into policy parameters.

Related papers

• ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers [58.411900827911346]
  We show that the missing ingredient is the treatment of residual connections.<n>We introduce Residual-aware Layer-wise Relevance Propagation (ResLRP)<n>ResLRP is a simple extension of LRP whose propagation rules explicitly account for cancellations in residual branches.
  arXiv  Detail & Related papers  (2026-09-15T13:17:30Z)
• ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human Feedback [1.711436622393758]
  Online Residual Policy Adaptation (ORPA) is a framework that enables feedback-driven correction of robot actions without modifying the underlying policy parameters.<n>ORPA augments a pretrained control policy with a lightweight, feedback-conditioned module that predicts residual adjustments directly in joint space.<n>We evaluate ORPA on a set of precision-sensitive manipulation tasks using the ALOHA platform, demonstrating improvements in success rate and recovery from small perturbations compared to baseline control policies and rule-based inverse kinematics corrections.
  arXiv  Detail & Related papers  (2026-08-18T03:30:12Z)
• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• OGPO: Sample Efficient Full-Finetuning of Generative Control Policies [53.42266064673132]
  Generative control policies (GCPs) have emerged as effective parameterizations for robot learning.<n>This work introduces Off-policy Generative Policy Optimization (OGPO), a sample-efficient for finetuning GCPs.<n>OGPO achieves state-of-the-art performance on manipulation tasks spanning multi-task settings, high-precision insertion, and dexterous control.
  arXiv  Detail & Related papers  (2026-05-04T18:36:40Z)
• Structure-Dependent Regret and Constraint Violation Bounds for Online Convex Optimization with Time-Varying Constraints [12.438794398358548]
  We introduce a structured characterization of constraint variation to common network phenomena.<n>We derive structure-dependent joint bounds that strictly improve upon adversarial rates when the constraint process exhibits regularity.<n>Experiments on synthetic benchmarks and real-world datasets demonstrate that SA-PD reduces cumulative constraint violation by up to 53%.
  arXiv  Detail & Related papers  (2026-03-15T10:30:35Z)
• Residual Control for Fast Recovery from Dynamics Shifts [1.9446811955039964]
  Unobserved dynamics shifts can experience substantial transient performance degradation in robotic systems.<n>We propose a stability-aligned residual control architecture in which a reinforcement learning policy trained under nominal dynamics remains fixed at deployment.<n>We show that the proposed method consistently reduces recovery time relative to frozen and online-adaptation baselines while maintaining near-nominal steady-state performance.
  arXiv  Detail & Related papers  (2026-03-08T19:33:33Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• Bayesian Conservative Policy Optimization (BCPO): A Novel Uncertainty-Calibrated Offline Reinforcement Learning with Credible Lower Bounds [1.2183405753834562]
  offline reinforcement learning (RL) aims to learn decision policies from a fixed batch of logged transitions.<n>We propose emphBayesian Conservative Policy Optimization (BCPO), a unified framework that converts uncertainty into emphprovably conservative policy improvement.<n>BCPO maintains a hierarchical Bayesian posterior over environment/value models, constructs a emphcredible lower bound (LCB) on action values, and performs policy updates under explicit KL regularization.
  arXiv  Detail & Related papers  (2026-03-06T01:46:02Z)
• When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift [64.37959940809633]
  We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures.<n>We show Transformer-based sequence policies substantially outperform, RNN, and SSMs in robustness, maintaining high returns even when large fractions of sensors are unavailable.
  arXiv  Detail & Related papers  (2026-03-04T22:21:54Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• ModalImmune: Immunity Driven Unlearning via Self Destructive Training [21.940530514137947]
  ModalImmune enforces modality immunity by intentionally collapsing selected modality information during training.<n> framework combines a spectrum-adaptive collapse regularizer, an information-gain guided controller for targeted interventions, curvature-aware gradient masking to stabilize destructive updates.<n> Empirical evaluation on standard multimodal benchmarks demonstrates that ModalImmune improves resilience to modality removal and corruption while retaining convergence stability and reconstruction capacity.
  arXiv  Detail & Related papers  (2026-02-18T05:35:32Z)
• Unifying Stable Optimization and Reference Regularization in RLHF [64.16830602324345]
  This paper introduces a unified regularization approach that balances objectives of preventing reward hacking and maintaining stable policy updates.<n>Our simple yet principled alignment objective yields a weighted supervised fine-tuning loss with a superior trade-off, which demonstrably improves both alignment results and implementation complexity.
  arXiv  Detail & Related papers  (2026-02-12T03:31:19Z)
• AdaptNC: Adaptive Nonconformity Scores for Uncertainty-Aware Autonomous Systems in Dynamic Environments [7.201566646241765]
  Conformal Prediction methods maintain target coverage by adaptively scaling the conformal threshold.<n>We show that this fixed geometry leads to highly conservative, volume-inefficient prediction regions when environments undergo structural shifts.<n>We propose textbfAdaptNC, a framework for the joint online adaptation of both the nonconformity score parameters and the conformal threshold.
  arXiv  Detail & Related papers  (2026-02-02T04:41:35Z)
• Unraveling the Hidden Dynamical Structure in Recurrent Neural Policies [26.411234883034407]
  Recurrent neural policies are widely used in partially observable control and meta-RL tasks.<n>We find that stable cyclic structures consistently emerge during interaction with the environment.<n>These findings offer new perspectives to explain many nice properties of recurrent policies.
  arXiv  Detail & Related papers  (2026-02-01T12:37:42Z)
• Entropy Ratio Clipping as a Soft Global Constraint for Stable Reinforcement Learning [49.92803982100042]
  We propose using the entropy ratio between the current and previous policies as a new global metric.<n>We introduce an textbfEntropy Ratio (ERC) mechanism that imposes bidirectional constraints on the entropy ratio.<n>This stabilizes policy updates at the global distribution level and compensates for the inability of PPO-clip to regulate probability shifts of un-sampled actions.
  arXiv  Detail & Related papers  (2025-12-05T10:26:32Z)
• TopoReformer: Mitigating Adversarial Attacks Using Topological Purification in OCR Models [2.8272661103123258]
  Adversarially perturbed images of text can cause OCR systems to produce misleading or incorrect transcriptions from seemingly invisible changes to humans.<n>Existing defenses, such as adversarial training, input preprocessing, or post-recognition correction, are often model-specific, computationally expensive, and affect performance on unperturbed inputs.<n>TopoReformer is introduced, a model-agnostic reformation pipeline that mitigates adversarial perturbations while preserving the structural integrity of text images.
  arXiv  Detail & Related papers  (2025-11-19T19:01:51Z)
• GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping [63.33669214116784]
  GRPO-Guard is a simple yet effective enhancement to existing GRPO frameworks.<n>It restores a balanced and step-consistent importance ratio, ensuring that PPO clipping properly constrains harmful updates.<n>It substantially mitigates implicit over-optimization without relying on heavy KL regularization.
  arXiv  Detail & Related papers  (2025-10-25T14:51:17Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learning [79.59753528758361]
  We present TrajBooster, a cross-embodiment framework that leverages abundant wheeled-humanoid data to boost bipedal VLA.<n>Our key idea is to use end-effector trajectories as a morphology-agnostic interface.<n>Results show that TrajBooster allows existing wheeled-humanoid data to efficiently strengthen bipedal humanoid VLA performance.
  arXiv  Detail & Related papers  (2025-09-15T12:25:39Z)
• Convergence and Generalization of Anti-Regularization for Parametric Models [0.0]
  Anti-regularization introduces a reward term with a reversed sign into the loss function.<n>We formalize spectral safety conditions and trust-region constraints.<n>We design a lightweight safeguard that combines a projection operator with gradient clipping to guarantee stable intervention.
  arXiv  Detail & Related papers  (2025-08-24T15:34:17Z)
• Non-stationary Online Learning with Memory and Non-stochastic Control [71.14503310914799]
  We study the problem of Online Convex Optimization (OCO) with memory, which allows loss functions to depend on past decisions. In this paper, we introduce dynamic policy regret as the performance measure to design algorithms robust to non-stationary environments. We propose a novel algorithm for OCO with memory that provably enjoys an optimal dynamic policy regret in terms of time horizon, non-stationarity measure, and memory length.
  arXiv  Detail & Related papers  (2021-02-07T09:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.