SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems

• URL: http://arxiv.org/abs/2608.22979v2
• Date: Tue, 25 Aug 2026 03:07:23 GMT
• Title: SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems
• Abstract summary: Hard quantization is compact but introduces boundary distortion, whereas dense soft quantization couples representation quality to the limited storage budget.<n>We propose Sparse-based Residual Soft Quantization (SA-RSQ), which uses Top-K sparse routing and softmax weights to store compact Activations.<n>Experiments on a proprietary food-delivery advertising dataset show favorable reconstruction-performance and CTR trade-offs across storage budgets of 8-48 bytes per item.
• Score: 10.538492036847094
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying high-dimensional multimodal features in industrial recommender systems incurs substantial storage and latency overhead. Hard quantization is compact but introduces boundary distortion, whereas dense soft quantization couples representation quality to the limited storage budget. We propose Sparse Activation-based Residual Soft Quantization (SA-RSQ), which uses Top-K sparse routing and softmax weights to store compact (Index, Probability) tuples. The stored tuples decouple per-item storage from codebook dimensionality; for a fixed selected support, gradients propagate through the routing weights and weighted reconstruction without relying on a straight-through estimator. Experiments on a proprietary food-delivery advertising dataset show favorable reconstruction-performance and CTR trade-offs across storage budgets of 8-48 bytes per item. A preliminary Next-Distribution Prediction study and a one-week online A/B test further demonstrate the practical potential of SA-RSQ, with relative lifts of +2.51% in CTR and +3.66% in CPM.

Related papers

• Hardware Robustness of Sample-Based Quantum Diagonalization [17.52301775749581]
  Quantum Diagonalization (SQD) is a hybrid quantum-classical method that replaces variational optimization with a self-consistent recovery loop over QPU samples.<n>We analyze SQD robustness on IBM Heron hardware across these dimensions.
  arXiv  Detail & Related papers  (2026-07-20T17:36:08Z)
• BitNet Text Embeddings [104.29781473344813]
  We present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage.<n> BITEMBED converts pretrained LLM backbones into BitNet-style embedding encoders with ternary weights, quantized activations, and lightweight normalization refinement.<n> Experiments on MMTEB (eng) with Qwen3-0.6B and Gemma703-2M show that BITEMBED is largely comparable to full precision teacher embedders.
  arXiv  Detail & Related papers  (2026-06-24T10:37:01Z)
• ConCise: Training-Free Conclusion-Chain State Compression for Cost-Efficient Multi-Step RAG Services [11.399162464271692]
  Multi-step retrieval-augmented generation (RAG) has been widely deployed as LLM-powered web services for complex question answering.<n>RAG causes historical documents and reasoning traces to accumulate across rounds, inflating cumulative input tokens.<n>This paper proposes ConCise, a training-free state-layer protocol that restructures cross-round context transmission.
  arXiv  Detail & Related papers  (2026-06-13T08:10:53Z)
• UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs [22.43695132973238]
  We introduce UniQL, a unified post-training quantization and low-rank compression framework with on-device pruning rates for edge LLMs.<n>UniQL is a general framework that integrates quantization and low-rank compression for Transformers, State Space Models (SSMs), and hybrid models.<n>Our framework performs weight-sorting, fine-tuning, and quantization in the cloud in a single-pass workflow, while enabling on-device pruning rates up to 35%.
  arXiv  Detail & Related papers  (2025-12-03T02:33:39Z)
• Quantized Visual Geometry Grounded Transformer [67.15451442018258]
  This paper proposes the first Quantization framework for VGGTs, namely QuantVGGT.<n>We introduce Dual-Smoothed Fine-Grained Quantization, which integrates pre-global Hadamard rotation and post-local channel smoothing.<n>We also design Noise-Filtered Diverse Sampling, which filters outliers via deep-layer statistics.
  arXiv  Detail & Related papers  (2025-09-25T15:17:11Z)
• A Universal Framework for Compressing Embeddings in CTR Prediction [68.27582084015044]
  We introduce a Model-agnostic Embedding Compression (MEC) framework that compresses embedding tables by quantizing pre-trained embeddings.<n>Our approach consists of two stages: first, we apply popularity-weighted regularization to balance code distribution between high- and low-frequency features.<n> Experiments on three datasets reveal that our method reduces memory usage by over 50x while maintaining or improving recommendation performance.
  arXiv  Detail & Related papers  (2025-02-21T10:12:34Z)
• BiMaCoSR: Binary One-Step Diffusion Model Leveraging Flexible Matrix Compression for Real Super-Resolution [63.777210548110425]
  We propose BiMaCoSR, which combines binarization and one-step distillation to obtain extreme compression and acceleration.<n>BiMaCoSR achieves a 23.8x compression ratio and a 27.4x speedup ratio compared to FP counterpart.
  arXiv  Detail & Related papers  (2025-02-01T06:34:55Z)
• PAMS: Quantized Super-Resolution via Parameterized Max Scale [84.55675222525608]
  Deep convolutional neural networks (DCNNs) have shown dominant performance in the task of super-resolution (SR) We propose a new quantization scheme termed PArameterized Max Scale (PAMS), which applies the trainable truncated parameter to explore the upper bound of the quantization range adaptively. Experiments demonstrate that the proposed PAMS scheme can well compress and accelerate the existing SR models such as EDSR and RDN.
  arXiv  Detail & Related papers  (2020-11-09T06:16:05Z)
• A Generic Network Compression Framework for Sequential Recommender Systems [71.81962915192022]
  Sequential recommender systems (SRS) have become the key technology in capturing user's dynamic interests and generating high-quality recommendations. We propose a compressed sequential recommendation framework, termed as CpRec, where two generic model shrinking techniques are employed. By the extensive ablation studies, we demonstrate that the proposed CpRec can achieve up to 4$sim$8 times compression rates in real-world SRS datasets.
  arXiv  Detail & Related papers  (2020-04-21T08:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.