CRMA: A Spectrally-Bounded Backbone for Modular Continual Fine-Tuning of LLMs

• URL: http://arxiv.org/abs/2606.00382v1
• Date: Fri, 29 May 2026 21:50:23 GMT
• Title: CRMA: A Spectrally-Bounded Backbone for Modular Continual Fine-Tuning of LLMs
• Abstract summary: We introduce CRMA, a residual adapter whose internal mixing matrix M is doubly-stochastic at every forward pass via Sinkhorn normalization.<n>On Mistral-7B across 5 domains 3 seeds, modular per-task LoRA mediates a backbone loss-relative drift from +42.96% +/- 5.5 (naive fine-tuning) to -0.17% +/- 0.17, with disjoint per-seed ranges, and improves prior-task holdout loss by 1.99% +/- 0.54 over a matched frozen-substrate baseline.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential fine-tuning of large language models forces a choice: let the shared substrate keep learning and accept catastrophic forgetting, or freeze it after task one and foreclose cross-task refinement. Per-task adapter methods (LoRAHub, AdapterFusion, PackNet, Progressive Networks) take the second path. We introduce CRMA (Constrained Residual Mixing Adapter), a residual adapter whose internal mixing matrix M is doubly-stochastic at every forward pass via Sinkhorn normalization, so by Birkhoff's theorem ||M||_2 <= 1 holds by construction -- a structural bound, not a penalty. CRMA's spectrally bounded backbone provides a continuously trained shared substrate that earlier modular methods could not, while preserving their forgetting guarantees. On Mistral-7B across 5 sequential domains and 3 seeds, modular per-task LoRA on a CRMA backbone reduces loss-relative drift from +42.96% +/- 5.5 (naive sequential fine-tuning) to -0.17% +/- 0.17, with disjoint per-seed ranges, and improves prior-task holdout loss by 1.99% +/- 0.54 over a matched frozen-substrate baseline. Three independent experimental setups (Mistral-7B 4-domain controlled ablation, TinyLlama 3-domain contamination-controlled replication, Mistral-7B cross-domain probes at 7B) all show positive backward transfer -- without replay buffers, without growing per-task memory, and without distillation. An inference-time ablation on Gemma-2-9B confirms CRMA mediates access to sequentially trained knowledge: 98/100 vs. 38/100 on the same weights and same questions with only CRMA injection toggled. 867 logged training steps verify ||M||_2 = 1.0 within float32 precision (max deviation 1.2 x 10^-7). The forgetting-prevention effect holds across 1.1B-9.2B parameters and four architecture families.

Related papers

• ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training [66.72708893922605]
  Post-training of large multimodal models should add new capabilities while preserving those from pre-training.<n>SFT gives explicit target supervision that learns a task from near-zero accuracy, but its off-policy targets move the model far enough to cause forgetting.<n>We introduce ReDraft, which obtains both from the model's own failures.
  arXiv  Detail & Related papers  (2026-09-15T04:59:03Z)
• Long-Range Indirect Control-Flow Prediction in Stripped Binaries via Dual Virtual Hubs and Multi-Task Graph Learning [44.79384874347874]
  We present ICFlowNet, a unified framework for long-range ICF prediction in stripped binaries.<n>ICFlowNet introduces candidate-aware Dual Virtual Hubs, a Global Code Hub and a Global Data Hub.<n>We also develop a leakage-aware, noise-controlled pipeline with package-level splits, function-level mnemonic-hash deduplication, and a clean test protocol.
  arXiv  Detail & Related papers  (2026-09-03T02:17:23Z)
• XMerge: Cross-Axis Selection and Reconstructive Layer Merging for LLM Depth Compression [0.0]
  Cross-axis selection identifies a block with low relative-magnitude and angular hidden-state change.<n>Local boundary reconstruction re-fits the adjacent surviving block to match the original two-block output.<n>XMerge uses no task labels or end-to-end fine-tuning.
  arXiv  Detail & Related papers  (2026-09-02T04:12:44Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• TALAN: Task-Aligned Latent Adaptation Networks for Targeted Post-Training of Large Language Models [8.47008876232157]
  We introduce TALAN (Task-Aligned Latent Adaptation Networks), a sequence-conditioned latent side path inserted into a transformer's residual stream.<n> TALAN compresses the active sequence into latent memory, remixes it into token-level perturbations, and writes them back through a controlled residual update.<n>Across four Qwen3-family backbones and four STEM/code benchmarks, TALAN improves matched LoRA and DoRA baselines.
  arXiv  Detail & Related papers  (2026-06-05T04:34:50Z)
• Reliable Extraction of Clinical Follow-Up Instructions: A Hybrid Neural-Symbolic Pipeline [43.0484058393522]
  We test a hybrid neural-symbolic pipeline against direct generation.<n>We evaluate on a 2,000-note synthetic outpatient corpus with action-disjoint splits.
  arXiv  Detail & Related papers  (2026-05-26T05:14:33Z)
• Decomposing Evolutionary Mixture-of-LoRA Architectures: The Routing Lever, the Lifecycle Penalty, and a Substrate-Conditional Boundary [3.151184728006369]
  We decompose an evolutionary mixtureofLoRA system into three factors -- a router rewrite, a perdomain leave-one-out evaluation scope, and a lifecycle of death plus alpha-blend inheritance plus SVD mutation plus slot reallocation.<n>A synthetic sandbox locates a substrate-conditional regime: evolutionary search on the routing channel is load-bearing only when are pre-aligned to the task; in every other regime tested it underperforms, ties, or actively degrades the solution.
  arXiv  Detail & Related papers  (2026-05-11T18:59:11Z)
• The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs [52.709361620508595]
  ListOPD brings a student to in-domain with an 8B-SFT baseline at one-fifth the parameters.<n>On Amazon Fashion, three pre-registered tests--a fine-grid cliff interval test, and a small-clip cross-prediction--fall within their locked prediction windows--with the small-clip value matching the closed-form prediction below grid resolution.
  arXiv  Detail & Related papers  (2026-05-09T06:48:00Z)
• SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models [48.335262141752715]
  Post-training pipeline for diffusion models has two stages: supervised fine-tuning (SFT) on curated data and reinforcement learning (RL) with reward models.<n>We propose SOAR (Self-Correction for Optimal Alignment and Refinement), a bias-correction post-training method that fills this gap.<n>It is on-policy, reward-free, and provides dense per-time supervision with no credit-assignment problem.
  arXiv  Detail & Related papers  (2026-04-14T11:45:15Z)
• Ouroboros: Dynamic Weight Generation for Recursive Transformers via Input-Conditioned LoRA Modulation [0.0]
  Ouroboros is a system that attaches a compact Controller hypernetwork to a recursive transformer block.<n>The Controller observes the current hidden state, produces a per-step diagonal modulation vector, and applies it to frozen SVD-d LoRA bases.<n>On Qwen2.5-3B split into a Prelude/Recurrent/Coda architecture, Ouroboros reduces training loss by 43.4% over the unmodified 17-layer baseline.<n>The full system adds only 9.2M trainable parameters (Controller, gate, and per-step norms) yet outperforms equivalently-sized static per-step LoRA by 1.44 loss
  arXiv  Detail & Related papers  (2026-04-02T13:52:32Z)
• S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models [0.0]
  tuning a single initial state matrix per recurrent layer with zero overhead outperforms LoRA by +10.8 pp (p wenSpider on HumanEval)<n> tuning method we call S0 tuning improves greedy pass@1 by +23.6 +/- 1.7 pp (10 seeds)<n>Cross-domain transfer is significant on MATH-500 (+4.8 pp, p = 0.00002, 8 seeds) and GSM8K = 0.0003, 10 seeds.
  arXiv  Detail & Related papers  (2026-04-01T17:21:15Z)
• Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes [3.305265383862785]
  We present an empirical study of Low-Rank Adaptation (LoRA) in sequential transformer encoder finetuning with companion representation probes.<n>In five full-validation BERT-base reruns on an RTE->MRPC->CoLA->SST-2 sequence, full fine-tuning yields 19.9% +/- average forgetting, whereas standard LoRA (r, query/value modules) yields 0.6% +/-1.4%.
  arXiv  Detail & Related papers  (2026-03-29T14:14:36Z)
• RPMS: Enhancing LLM-Based Embodied Planning through Rule-Augmented Memory Synergy [34.723296971298424]
  LLM agents often fail in closed-world embodied environments because actions must satisfy strict preconditions.<n>We identify two structurally coupled failure modes: (P1) invalid action generation and (P2) state drift.<n>We present RPMS, a conflict-managed architecture that enforces action feasibility via structured rule retrieval.
  arXiv  Detail & Related papers  (2026-03-18T15:26:00Z)
• SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud [44.19683744405876]
  Embodied AI requires sub-second inference near the Radio Access Network (RAN)<n>We report measurements from a 5G Standalone (SA) AI-RAN testbed using a fixed baseline policy for repeatability.
  arXiv  Detail & Related papers  (2026-02-27T06:43:47Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets [118.22975463000928]
  We consider an offline reinforcement learning (RL) setting where the agent need to learn from a dataset collected by rolling out multiple behavior policies. There are two challenges for this setting: 1) The optimal trade-off between optimizing the RL signal and the behavior cloning (BC) signal changes on different states due to the variation of the action coverage induced by different behavior policies. In this paper, we address both challenges by using adaptively weighted reverse Kullback-Leibler (KL) divergence as the BC regularizer based on the TD3 algorithm.
  arXiv  Detail & Related papers  (2022-12-05T09:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.