LoRA-RC: Reservoir Computing with Low-Rank Adaptation

• URL: http://arxiv.org/abs/2609.12327v1
• Date: Fri, 11 Sep 2026 01:19:48 GMT
• Title: LoRA-RC: Reservoir Computing with Low-Rank Adaptation
• Abstract summary: LoRA-RC adapts the recurrent matrix through a low-rank correction driven by streaming prediction errors.<n>On a Lorenz system with an abrupt parameter drift, LoRA-RC cuts post-drift prediction error by 56% versus a fixed RC and 51% versus readout-only adaptation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reservoir computing (RC) trains only a linear readout over a fixed recurrent layer, making it fast and data-efficient for online prediction. However, a static reservoir degrades under system drift, readout-only adaptation is then insufficient, and unconstrained reservoir adaptation can destroy the echo-state and incremental stability properties that make RC reliable. This paper proposes LoRA-RC, which adapts the recurrent matrix through a low-rank correction driven by streaming prediction errors. The base reservoir and adaptation bases are fixed offline; a small core matrix is adapted online, projected onto a spectral-norm ball, and low-pass filtered at each step. The projection guarantees that every applied recurrent matrix remains within a certified contraction set, and an incremental input-to-state stability bound is established for the reservoir along each online adaptation path, with path-independent rate and gain. On a Lorenz system with an abrupt parameter drift, LoRA-RC cuts post-drift prediction error by 56% versus a fixed RC and 51% versus readout-only adaptation; ablations over 20 seeds show that removing the projection inflates this error by more than a factor of 40.

Related papers

• SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs [18.494055578192015]
  We introduce SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter at each pruning site.<n>Each LRA preserves the identity pathway of the original residual block and adds a lightweight affine residual correction.<n>Experiments on five model families, six layer-selection criteria, and seven zero-shot benchmarks show that SHIFT-LLM consistently recovers accuracy lost to depth pruning.
  arXiv  Detail & Related papers  (2026-08-25T19:01:58Z)
• SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation [64.33799467286265]
  Low-Rank Adaptation (LoRA) enables efficient adaptation of large pre-trained models to downstream tasks.<n>We show that when a full fine-tuning gradient is backpropagated to the low-rank matrices, it undergoes anisotropic scaling driven by their singular values.<n>We propose a new low-rank parameterization, SDS-LoRA, which structurally decouples singular values from the backward pass.
  arXiv  Detail & Related papers  (2026-06-15T09:27:50Z)
• HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization [48.87937677659571]
  We introduce HARP (Hadamard-preconditioned Adaptive Rotation Processor), a learnable structured two-sided processor that replaces fixed Hadamard mixing.<n>Across 2-4 bit settings on models ranging from 1B to 70B parameters, HARP improves perplexity and zero-shot accuracy over fixed RHT.<n>HarP preserves deployment efficiency, reaching 128 tok/s versus 61 tok/s for FP16.
  arXiv  Detail & Related papers  (2026-05-28T12:24:15Z)
• Trust Region Q Adjoint Matching [54.05514246126841]
  We introduce Trust Region Q-Adjoint Matching (TRQAM), a stable off-policy fine-tuning algorithm that adaptively controls the path-space KL with pretrained flow policies.<n> TRQAM consistently outperforms prior arts in both offline RL and offline-to-online RL.
  arXiv  Detail & Related papers  (2026-05-26T14:28:43Z)
• Relative Repairability: A Calibration-Based Diagnostic for High-Sparsity Post-Pruning Allocation [12.288779306467768]
  Relative Repairability (RR) is a calibration based diagnostic that compares the raw activation distortion caused by layerwise pruning with the residual distortion left after channelwise variance matching repair.<n>RR is most useful near an architecture dependent recoverability transition, where standard structural or magnitude based allocation priors begin to lose reliability but post repair recovery has not yet fully collapsed.<n>These results suggest that high sparsity pruning should allocate not only retained weights, but also repairable damage.
  arXiv  Detail & Related papers  (2026-05-25T07:12:38Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• ARC-STAR: Auditable Post-Hoc Correction for PDE Foundation Models [47.9430487327673]
  ARC-STAR organizes correction into three stages: a global corrector removes broad solver bias, a blockwise local refiner cleans the post-global residual, and, at deployment, a label-free score routes refinement to high-risk blocks under a compute budget.<n>ARC-STAR is the only method that cuts velocity rollout error by at least 36x over raw Poseidon on every cell.
  arXiv  Detail & Related papers  (2026-05-21T09:26:16Z)
• Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks [8.5133415329194]
  One-shot magnitude pruning can cause severe accuracy collapse in the high-sparsity regime.<n>We propose Adaptive Signal Resuscitation (ASR) as a training-free channel-wise repair method.<n>ASR estimates a variance-matching correction for each output channel and stabilizes it with a data-driven shrinkage rule.
  arXiv  Detail & Related papers  (2026-05-20T17:19:01Z)
• Reframing preprocessing selection as model-internal calibration in near-infrared spectroscopy: A large-scale benchmark of operator-adaptive PLS and Ridge models [0.2609784101826761]
  This paper studies the case where that search can be collapsed into one calibration step.<n> Linear operator-adaptive calibration gives comparable prediction quality to exhaustive preprocessing screening.
  arXiv  Detail & Related papers  (2026-05-13T14:23:00Z)
• OCRR: A Benchmark for Online Correction Recovery under Distribution Shift [0.0]
  OCRR (Online Correction Recovery Rate) is a benchmark that streams a corpus through a classification system and applies oracle or corrections to wrong predictions.<n>It beats the next-best published continual-learning baseline by 32.6 percentage points at equal memory budget.<n>It also beats LoRA-on-DeBERTa-v3-large by 84.6 percentage points on retention.
  arXiv  Detail & Related papers  (2026-05-04T20:51:07Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.