QUADS: QUAntized Distillation Framework for Efficient Speech Language Understanding

• URL: http://arxiv.org/abs/2505.14723v1
• Date: Mon, 19 May 2025 16:09:51 GMT
• Title: QUADS: QUAntized Distillation Framework for Efficient Speech Language Understanding
• Abstract summary: Existing methods apply distillation and quantization separately, leading to suboptimal compression.<n>We propose QUADS, a unified framework that optimize both through multi-stage training with a pre-tuned model.<n>QUADS achieves 71.13% accuracy on SLURP and 99.20% on FSC, with only minor degradations of up to 5.56% compared to state-of-the-art models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spoken Language Understanding (SLU) systems must balance performance and efficiency, particularly in resource-constrained environments. Existing methods apply distillation and quantization separately, leading to suboptimal compression as distillation ignores quantization constraints. We propose QUADS, a unified framework that optimizes both through multi-stage training with a pre-tuned model, enhancing adaptability to low-bit regimes while maintaining accuracy. QUADS achieves 71.13\% accuracy on SLURP and 99.20\% on FSC, with only minor degradations of up to 5.56\% compared to state-of-the-art models. Additionally, it reduces computational complexity by 60--73$\times$ (GMACs) and model size by 83--700$\times$, demonstrating strong robustness under extreme quantization. These results establish QUADS as a highly efficient solution for real-world, resource-constrained SLU applications.

Related papers

• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• QuantSR+: Pushing the Limit of Quantized Image Super-Resolution Networks [63.04549067734431]
  Low-bit quantization is widely used to compress super-resolution (SR) models.<n>However, when SR models are pushed to ultra-low precision (2-4 bits), performance can drop sharply.<n>We propose QuantSR+, a unified framework that improves quantization operators, network design, and training optimization.
  arXiv  Detail & Related papers  (2026-05-21T11:38:52Z)
• Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference [6.886536285117155]
  Budgeted LoRA is a framework that treats model compression as a structured compute allocation problem.<n>We show that Budgeted LoRA matches standard LoRA perplexity at a moderate budget with a 1.74x compressed-module speedup.<n>It achieves a 4.05x speedup with moderate perplexity degradation, and it preserves higher accuracy on function-style in-context learning probes.
  arXiv  Detail & Related papers  (2026-05-05T22:59:14Z)
• Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation [46.34608916687127]
  Low-Rank Decomposed Scaling (LoRDS) is a unified framework that rethinks quantization granularity through this low-rank decomposition.<n>By "breaking the blocks" of spatial constraints, LoRDS establishes a seamless efficiency lifecycle.<n>LoRDS consistently outperforms state-of-the-art baselines across various model families in both quantization and downstream fine-tuning tasks.
  arXiv  Detail & Related papers  (2026-01-30T08:46:02Z)
• Punching Above Precision: Small Quantized Model Distillation with Learnable Regularizer [9.85847764731154]
  Game of Regularizer (GoR) is a learnable regularization method that adaptively balances task-specific (TS) and distillation losses.<n>GoR consistently outperforms state-of-the-art QAT-KD methods on low-power edge devices.<n>We also introduce QAT-EKD-GoR, an ensemble distillation framework that uses multiple heterogeneous teacher models.
  arXiv  Detail & Related papers  (2025-09-25T07:43:13Z)
• First-Order Error Matters: Accurate Compensation for Quantized Large Language Models [32.69069234109942]
  Post-training quantization (PTQ) offers an efficient approach to compressing large language models (LLMs)<n>Existing compensation-based weight calibration methods often rely on a second-order Taylor expansion to model quantization error.<n>We propose FOEM, a novel PTQ method that explicitly incorporates first-order gradient terms to improve quantization error compensation.
  arXiv  Detail & Related papers  (2025-07-15T06:18:46Z)
• A High-Performance Multilevel Framework for Quantum Layout Synthesis [5.673807948689425]
  We propose ML-SABRE, a high-performance multilevel framework for QLS.<n>We employ the state-of-the-art method, LightSABRE, at all levels to ensure both efficiency and performance.<n> ML-SABRE can significantly reduce the optimality gap by up to 82% for SWAP count and 49% for circuit depth.
  arXiv  Detail & Related papers  (2025-05-30T03:24:22Z)
• Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis [9.884521812433661]
  Quaff is a Quantized parameter-efficient fine-tuning framework for large language models.<n>It suppresses outliers exclusively invariant channels using lightweight operations.<n>It achieves a 1.73x latency reduction and 30% memory savings over full-precision fine-tuning.
  arXiv  Detail & Related papers  (2025-05-20T07:19:36Z)
• Optimizing Large Language Model Training Using FP4 Quantization [73.55459961002371]
  Quantized training presents a promising solution by enabling low-bit arithmetic operations to reduce costs.<n>This work introduces the first FP4 training framework for large language models (LLMs)
  arXiv  Detail & Related papers  (2025-01-28T18:04:50Z)
• FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing [59.12511498024836]
  We present a method to prune large language models (LLMs) that selectively prunes model blocks based on an importance score.<n>We propose a principled metric to replace each pruned block using a weight-sharing mechanism.<n> Empirical evaluations demonstrate substantial performance gains over existing methods.
  arXiv  Detail & Related papers  (2025-01-24T18:46:37Z)
• SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression [7.6131620435684875]
  SLIM is a new one-shot compression framework that holistically integrates hardware-friendly quantization, sparsity, and low-rank approximation.<n>SLIM improves model accuracy by up to 5.66% (LLaMA-2-7B) for 2:4 sparsity with 4-bit weight quantization, outperforming prior methods.<n>We also propose an optional PEFT recipe that further improves accuracy by up to 1.66% (LLaMA-2-13B) compared to SLIM without fine-tuning.
  arXiv  Detail & Related papers  (2024-10-12T18:36:07Z)
• Pruning Large Language Models with Semi-Structural Adaptive Sparse Training [17.381160429641316]
  Adaptive Sparse Trainer (AST) is a novel and efficient retraining framework tailored for semi-structured sparse models.<n>AST reduces the perplexity and zero-shot accuracy gap between dense and 2:4 semi-structured sparse models to 0.6 and 1.16%, respectively.
  arXiv  Detail & Related papers  (2024-07-30T06:33:44Z)
• EfficientQAT: Efficient Quantization-Aware Training for Large Language Models [50.525259103219256]
  quantization-aware training (QAT) offers a solution by reducing memory consumption through low-bit representations with minimal accuracy loss.<n>We propose Efficient Quantization-Aware Training (EfficientQAT), a more feasible QAT algorithm.<n> EfficientQAT involves two consecutive phases: Block-wise training of all parameters (Block-AP) and end-to-end training of quantization parameters (E2E-QP)
  arXiv  Detail & Related papers  (2024-07-10T17:53:30Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [67.67135738642547]
  Post-training quantization (PTQ) is a powerful compression technique investigated in large language models (LLMs) Existing PTQ methods are not ideal in terms of accuracy and efficiency, especially with below 4 bit-widths. This paper presents a Salience-Driven Mixed-Precision Quantization scheme for LLMs, namely SliM-LLM.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• Entanglement Distribution Delay Optimization in Quantum Networks with Distillation [51.53291671169632]
  Quantum networks (QNs) distribute entangled states to enable distributed quantum computing and sensing applications. QS resource allocation framework is proposed to enhance the end-to-end (e2e) fidelity and satisfy minimum rate and fidelity requirements.
  arXiv  Detail & Related papers  (2024-05-15T02:04:22Z)
• Automatic Mixed-Precision Quantization Search of BERT [62.65905462141319]
  Pre-trained language models such as BERT have shown remarkable effectiveness in various natural language processing tasks. These models usually contain millions of parameters, which prevents them from practical deployment on resource-constrained devices. We propose an automatic mixed-precision quantization framework designed for BERT that can simultaneously conduct quantization and pruning in a subgroup-wise level.
  arXiv  Detail & Related papers  (2021-12-30T06:32:47Z)
• Training with Quantization Noise for Extreme Model Compression [57.51832088938618]
  We tackle the problem of producing compact models, maximizing their accuracy for a given model size. A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator. In this paper, we extend this approach to work beyond int8 fixed-point quantization with extreme compression methods.
  arXiv  Detail & Related papers  (2020-04-15T20:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.