Gradient-Based LoRA Rank Allocation Under GRPO: An Empirical Study

• URL: http://arxiv.org/abs/2605.07366v1
• Date: Fri, 08 May 2026 07:22:38 GMT
• Title: Gradient-Based LoRA Rank Allocation Under GRPO: An Empirical Study
• Abstract summary: We investigate whether adaptive rank allocation for LoRA transfers to reinforcement learning.<n>We find proportional rank allocation degrades accuracy by 4.5 points compared to uniform allocation.<n>We identify two mechanisms behind this failure.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adaptive rank allocation for LoRA, allocating more parameters to important layers and fewer to unimportant ones, consistently improves efficiency under supervised fine-tuning (SFT). We investigate whether this success transfers to reinforcement learning, specifically Group Relative Policy Optimization (GRPO). Using gradient-magnitude profiling on Qwen 2.5 1.5B with GSM8K, we find that it does not: proportional rank allocation degrades accuracy by 4.5 points compared to uniform allocation (70.0% vs. 74.5%), despite using identical parameter budgets. We identify two mechanisms behind this failure. First, the gradient landscape under GRPO is fundamentally flatter than under SFT, the max-to-min layer importance ratio is only 2.17x, compared to >10x reported in SFT literature. All layers carry meaningful gradient signal; none are truly idle. Second, we discover a gradient amplification effect: non-uniform allocation widens the importance spread from 2.17x to 3.00x, creating a positive feedback loop where high-rank layers absorb more gradient while low-rank layers are progressively silenced. Our results suggest that gradient importance does not predict capacity requirements under RL, and that naive transfer of SFT-era rank allocation to alignment training should be avoided.

Related papers

• Between Gradient and Natural Gradient: A Continuum of LoRA Initializations [0.0]
  Low-rank adaptation (LoRA) fine-tunes large pretrained models at a fraction of the cost of full fine-tuning.<n>Recent schemes initialize adapters from the downstream loss gradient.<n>We show that these seemingly distinct methods are points on a single continuum.
  arXiv  Detail & Related papers  (2026-07-28T20:32:04Z)
• CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry [34.36337082439934]
  Deep Transformers are composed of uniformly stacked residual blocks, yet their deepest layers add little value.<n>We present two efficiency methods that exploit this asymmetry.<n> CascadeFormer tapers width with depth to match the uneven information flow across layers, achieving comparable perplexity to a uniform baseline at the same training budget while reducing latency by 8.6% and increasing throughput by 9.4%.
  arXiv  Detail & Related papers  (2026-06-25T02:25:00Z)
• SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation [64.33799467286265]
  Low-Rank Adaptation (LoRA) enables efficient adaptation of large pre-trained models to downstream tasks.<n>We show that when a full fine-tuning gradient is backpropagated to the low-rank matrices, it undergoes anisotropic scaling driven by their singular values.<n>We propose a new low-rank parameterization, SDS-LoRA, which structurally decouples singular values from the backward pass.
  arXiv  Detail & Related papers  (2026-06-15T09:27:50Z)
• Not How Many, But Which: Parameter Placement in Low-Rank Adaptation [7.686602368047758]
  Given a fixed budget of $k$ trainable entries within the B matrix of a LoRA adapter (A frozen), does the choice of which $k$ matter?<n>Under supervised fine-tuning, random and informed subsets achieve comparable performance.<n>Under GRPO on base models, random placement fails to improve over the base model, while gradient-informed placement recovers standard LoRA accuracy.
  arXiv  Detail & Related papers  (2026-05-12T14:46:00Z)
• Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures [0.0]
  Low-Rank Adaptation (LoRA) has become the dominant parameter-efficient fine-tuning method for large models.<n>We introduce Aletheia, a gradient-guided layer selection method.<n>We show that Aletheia can make LoRA fine-tuning materially more efficient without introducing major downstream damage on the evaluated set.
  arXiv  Detail & Related papers  (2026-04-04T10:24:12Z)
• IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring [9.250460219785188]
  We introduce IGU-LoRA, an adaptive-rank LoRA that computes within-layer Integrated Gradients (IG) sensitivities and aggregates them into a layer-level score for rank allocation.<n>IGU-LoRA consistently outperforms strong PEFT baselines at matched parameter budgets, improving downstream accuracy and robustness.
  arXiv  Detail & Related papers  (2026-03-14T06:45:54Z)
• GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning [55.03441672267886]
  We propose GradAlign, a gradient-aligned data selection method for reinforcement learning.<n>We evaluate GradAlign across three data regimes: unreliable reward signals, distribution imbalance, and low-utility training corpus.
  arXiv  Detail & Related papers  (2026-02-25T01:54:50Z)
• GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs [10.61152477422108]
  GradPruner can prune layers of Large Language Models guided by gradients in the early stages of fine-tuning.<n>Results demonstrate that GradPruner has achieved a parameter reduction of 40% with only a 0.99% decrease in accuracy.
  arXiv  Detail & Related papers  (2026-01-27T11:41:26Z)
• GAQAT: gradient-adaptive quantization-aware training for domain generalization [54.31450550793485]
  We propose a novel Gradient-Adaptive Quantization-Aware Training (GAQAT) framework for DG.<n>Our approach begins by identifying the scale-gradient conflict problem in low-precision quantization.<n>Extensive experiments validate the effectiveness of the proposed GAQAT framework.
  arXiv  Detail & Related papers  (2024-12-07T06:07:21Z)
• Gradient Normalization Provably Benefits Nonconvex SGD under Heavy-Tailed Noise [60.92029979853314]
  We investigate the roles of gradient normalization and clipping in ensuring the convergence of Gradient Descent (SGD) under heavy-tailed noise. Our work provides the first theoretical evidence demonstrating the benefits of gradient normalization in SGD under heavy-tailed noise. We introduce an accelerated SGD variant incorporating gradient normalization and clipping, further enhancing convergence rates under heavy-tailed noise.
  arXiv  Detail & Related papers  (2024-10-21T22:40:42Z)
• Class Gradient Projection For Continual Learning [99.105266615448]
  Catastrophic forgetting is one of the most critical challenges in Continual Learning (CL) We propose Class Gradient Projection (CGP), which calculates the gradient subspace from individual classes rather than tasks.
  arXiv  Detail & Related papers  (2023-11-25T02:45:56Z)
• Enabling Differentially Private Federated Learning for Speech Recognition: Benchmarks, Adaptive Optimizers and Gradient Clipping [42.2819343711085]
  We show that FL with DP is viable under strong privacy guarantees, provided a population of at least several million users.<n>We achieve user-level (7.2, $10-9$)-DP (resp. (4.5, $10-9$)-DP) with only a 1.3% absolute drop in word error rate when extrapolating to high (resp. low) population scales for FL with DP in ASR.
  arXiv  Detail & Related papers  (2023-09-29T19:11:49Z)
• LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning [56.88751562302793]
  Low-rank adaption (LoRA) has emerged to fine-tune large language models (LLMs) LoRAPrune is a new framework that delivers an accurate structured pruned model in a highly memory-efficient manner. LoRAPrune achieves a reduction in perplexity by 4.81 on WikiText2 and 3.46 on PTB, while also decreasing memory usage by 52.6%.
  arXiv  Detail & Related papers  (2023-05-28T15:15:48Z)
• The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition [84.51875325962061]
  We propose a gradient-driven training mechanism to tackle the long-tail problem. We introduce a new family of gradient-driven loss functions, namely equalization losses. Our method consistently outperforms the baseline models.
  arXiv  Detail & Related papers  (2022-10-11T16:00:36Z)
• The Break-Even Point on Optimization Trajectories of Deep Neural Networks [64.7563588124004]
  We argue for the existence of the "break-even" point on this trajectory. We show that using a large learning rate in the initial phase of training reduces the variance of the gradient. We also show that using a low learning rate results in bad conditioning of the loss surface even for a neural network with batch normalization layers.
  arXiv  Detail & Related papers  (2020-02-21T22:55:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.