How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model

• URL: http://arxiv.org/abs/2607.25583v1
• Date: Tue, 28 Jul 2026 11:12:03 GMT
• Title: How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
• Abstract summary: We show that LoRA with r=16 recovers within 11.6 percentage points of full fine-tuning accuracy (59.6% vs. 71.2% exact-match) while training fewer than 1% of parameters and consuming 31% less peak GPU memory.<n>QLoRA with INT8 and NF4 quantization achieves comparable accuracy (52.8% and 53.2%) at dramatically lower memory cost (0.60 GB each)
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Parameter-efficient fine-tuning (PEFT) and low-bit quantization are now standard tools for adapting language models under tight compute budgets, yet their interaction is most often studied on billion-parameter models where the design space is expensive to explore. We ask a complementary question: on a specific, fully reproducible 60M-parameter encoder-decoder model (T5-small) and a single-table text-to-SQL benchmark (WikiSQL), how much task accuracy does each efficiency knob actually cost? We run a controlled, single-variable study over (i) LoRA rank r in {2, 4, 8, 16, 32}, (ii) the set of adapted modules, and (iii) numerical precision. We report task accuracy alongside system-level metrics including trainable parameters, peak training memory, inference latency, and throughput, and frame adaptation as a constrained trade-off rather than an accuracy-only objective. Our results show that LoRA with r=16 recovers within 11.6 percentage points of full fine-tuning accuracy (59.6% vs. 71.2% exact-match) while training fewer than 1% of parameters and consuming 31% less peak GPU memory. Within this setting, rank beyond r=16 yields no measurable accuracy gain. QLoRA with INT8 and NF4 quantization achieves comparable accuracy (52.8% and 53.2%) at dramatically lower memory cost (0.60 GB each), demonstrating a compelling trade-off for memory-constrained deployments. All code, configurations, and logs are released for full reproducibility.

Related papers

• LoRA Fine-Tuned Models for Control Systems Course Q\&A: A Multidimensional Evaluation of Model Scale and Rank Effects [12.865623591533456]
  Large language models (LLMs) are increasingly used in specialized university courses.<n>Direct general-purpose responses may be inconsistently structured and hard to verify.<n>We built a supervised fine-tuning dataset of 360 system-user-assistant conversations.
  arXiv  Detail & Related papers  (2026-09-12T13:04:04Z)
• Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models [0.0]
  We introduce Variable Bit-width Quantization (VBQ)<n>Training-time method in which each contiguous group of 64 weights learns its own resolution from 1,2,4,8 bits.<n>As quality-per-byte, VBQ is 3.9-8.4x more efficient than FP16.
  arXiv  Detail & Related papers  (2026-07-03T02:42:04Z)
• FDM: A Framework for Decision-making to build ML-based Malware detection systems [3.552368449344917]
  This paper proposes the Framework for Decision-making (FDM) to build ML-based malware detection systems.<n>The FDM ranks five operational parameters to ranked recommendations across nine configuration dimensions.<n>Four experiments were conducted on three datasets to validate the framework.
  arXiv  Detail & Related papers  (2026-06-05T04:21:22Z)
• GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets [28.100700446484595]
  Mixed-precision quantization improves the budget--accuracy trade-off for large language models.<n>We propose GAMMA, a framework that learns module-wise precision preferences entirely within a post-training pipeline.
  arXiv  Detail & Related papers  (2026-05-18T14:30:58Z)
• One-for-All: A Lightweight Stabilized and Parameter-Efficient Pre-trained LLM for Time Series Forecasting [4.364999214109123]
  One-for-All is a framework for adapting pre-trained Large Language Models for time-series analysis.<n>rsLoRA introduces a mathematically grounded rank-stabilization mechanism that enables provable gradient stability at low ranks.<n>One-for-All achieves state-of-the-art efficiency-accuracy trade-offs.
  arXiv  Detail & Related papers  (2026-03-31T13:54:43Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• EfficientQAT: Efficient Quantization-Aware Training for Large Language Models [50.525259103219256]
  quantization-aware training (QAT) offers a solution by reducing memory consumption through low-bit representations with minimal accuracy loss.<n>We propose Efficient Quantization-Aware Training (EfficientQAT), a more feasible QAT algorithm.<n> EfficientQAT involves two consecutive phases: Block-wise training of all parameters (Block-AP) and end-to-end training of quantization parameters (E2E-QP)
  arXiv  Detail & Related papers  (2024-07-10T17:53:30Z)
• QLoRA: Efficient Finetuning of Quantized LLMs [66.58009990713134]
  We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU. QLoRA backpropagates through a frozen, 4-bit quantized pretrained language model into Low Rank Adapters(LoRA) Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark.
  arXiv  Detail & Related papers  (2023-05-23T17:50:33Z)
• The case for 4-bit precision: k-bit Inference Scaling Laws [75.4335600212427]
  Quantization methods reduce the number of bits required to represent each parameter in a model. The final model size depends on both the number of parameters of the original model and the rate of compression. We run more than 35,000 zero-shot experiments with 16-bit inputs and k-bit parameters to examine which quantization methods improve scaling for 3 to 8-bit precision.
  arXiv  Detail & Related papers  (2022-12-19T18:48:33Z)
• Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems [63.713297451300086]
  We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B. Their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system.
  arXiv  Detail & Related papers  (2022-06-15T20:44:23Z)
• LoRA: Low-Rank Adaptation of Large Language Models [71.75808607987281]
  Low-Rank Adaptation, or LoRA, freezes the pre-trained model weights and injects trainable rank decomposition into each layer of the Transformer architecture. For GPT-3, LoRA can reduce the number of trainable parameters by 10,000 times and the computation hardware requirement by 3 times compared to full fine-tuning.
  arXiv  Detail & Related papers  (2021-06-17T17:37:18Z)
• HAWQV3: Dyadic Neural Network Quantization [73.11579145354801]
  Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values. We present HAWQV3, a novel mixed-precision integer-only quantization framework.
  arXiv  Detail & Related papers  (2020-11-20T23:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.