LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression

• URL: http://arxiv.org/abs/2608.26389v1
• Date: Wed, 26 Aug 2026 20:27:21 GMT
• Title: LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
• Abstract summary: LowRankArena is a standardized evaluation platform for SVD-based LLM compression.<n>LowRankArena unifies task versions, uniform-precision compression budgets, comparison regimes, and inference measurements.<n>Our aligned audit of five representative SVD methods reveals that prior findings are highly conditional under standardized protocols.
• Score: 39.171753509551145
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison across existing studies remains difficult as prior evaluations use varied benchmarks, inconsistent ratios, and diverse setups, often failing to isolate low-rank effects from auxiliary techniques. As a result, it remains unclear whether reported gains reflect method-level improvements or differences in evaluation protocol. This lack of comparability highlights the need for a unified, reproducible evaluation platform. To address this problem, we present LowRankArena, a standardized evaluation platform for SVD-based LLM compression. LowRankArena unifies task versions, uniform-precision compression budgets, comparison regimes, and inference measurements, and provides a reproducible pipeline with over 3 TiB released compressed checkpoints. Using LowRankArena, our aligned audit of five representative SVD methods reveals that prior findings are highly conditional under standardized protocols: clear leaders and performance tiers shift across backbones and keep ratios, multiple-choice accuracy can hide large perplexity degradation, and nominal low-rank savings yield workload-dependent and often limited end-to-end speedups. Our code is available at: https://github.com/Zishan-Shao/lowrankarena.git.

Related papers

• LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression [4.953709812007795]
  We propose LACE-SVD, a Loss-Aware SVD framework with Cumulative Error correction for LLM compression.<n>LACE-SVD first estimates the calibration negative-log-likelihood increase induced by candidate layer-wise compression ratios.<n>It then refines the compressed model with closed-form local updates and introduces a propagation-aware correction for residual-stream output modules.
  arXiv  Detail & Related papers  (2026-07-03T07:46:13Z)
• REVES: REvision and VErification--Augmented Training for Test-Time Scaling [53.197756110943395]
  We propose a two-stage iterative framework that alternates between online data/prompt augmentation and policy optimization.<n>We observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training.
  arXiv  Detail & Related papers  (2026-06-17T10:37:23Z)
• IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression [21.844141881475533]
  SVD-based post-training compression offers a hardware-agnostic way to reduce model size and improve inference efficiency.<n>Existing methods often rely on input-only whitening spaces, homogeneous rank allocation, or loss-agnostic allocations.<n>We propose Input-Output Whitened SVD (IO-SVD), a post-training compression method that forms a KL-aware double-sided whitening space for model weights.
  arXiv  Detail & Related papers  (2026-05-15T05:19:10Z)
• Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression [11.908793753919745]
  We propose textbfZero Sum SVD (textbfZS-SVD), a post-training method that performs singular component selection in whitened coordinates.<n>textbfZS-SVD prunes components across the whole model with a textbfzero sum rule that keeps the cumulative predicted loss change near zero.<n>Experiments show consistent gains across diverse benchmarks and compression ratios.
  arXiv  Detail & Related papers  (2026-02-02T21:51:01Z)
• Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods [54.4711434793961]
  We show that simple image downsampling consistently outperforms many advanced compression methods across multiple widely used benchmarks.<n>Motivated by these findings, we introduce VTC-Bench, an evaluation framework that incorporates a data filtering mechanism to denoise existing benchmarks.
  arXiv  Detail & Related papers  (2025-10-08T15:44:28Z)
• QuantVSR: Low-Bit Post-Training Quantization for Real-World Video Super-Resolution [53.13952833016505]
  We propose a low-bit quantization model for real-world video super-resolution (VSR)<n>We use a calibration dataset to measure both spatial and temporal complexity for each layer.<n>We refine the FP and low-bit branches to achieve simultaneous optimization.
  arXiv  Detail & Related papers  (2025-08-06T14:35:59Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• AdaSVD: Adaptive Singular Value Decomposition for Large Language Models [75.1196637934987]
  Singular Value Decomposition (SVD) has emerged as a promising compression technique for large language models (LLMs)<n>Existing SVD-based methods often struggle to effectively mitigate the errors introduced by SVD truncation.<n>We propose AdaSVD, an adaptive SVD-based LLM compression approach.
  arXiv  Detail & Related papers  (2025-02-03T14:34:37Z)
• SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression [14.818355326032538]
  Singular Value Decomposition (SVD) offers a promising solution for Large Language Models (LLMs) compression.<n>However, truncating smaller singular values may lead to higher compression loss, and the lack of update on the compressed weights after SVD truncation.<n>We propose SVD-LLM, a SVD-based post-training LLM compression method that addresses the limitations of existing methods.
  arXiv  Detail & Related papers  (2024-03-12T07:31:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.