Continuous Input Embedding Size Search For Recommender Systems

• URL: http://arxiv.org/abs/2304.03501v5
• Date: Thu, 23 Oct 2025 05:52:33 GMT
• Title: Continuous Input Embedding Size Search For Recommender Systems
• Authors: Yunke Qu, Tong Chen, Xiangyu Zhao, Lizhen Cui, Kai Zheng, Hongzhi Yin,
• Abstract summary: Continuous input embedding size search (CIESS) is a novel RL-based method that operates on a continuous search space with arbitrary embedding sizes to choose from.<n> CIESS is also model-agnostic and hence generalizable to a variety of latent factor RSs.<n> experiments on two real-world datasets have shown state-of-the-art performance of CIESS under different memory budgets.
• Score: 60.89189829112067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Latent factor models are the most popular backbones for today's recommender systems owing to their prominent performance. Latent factor models represent users and items as real-valued embedding vectors for pairwise similarity computation, and all embeddings are traditionally restricted to a uniform size that is relatively large (e.g., 256-dimensional). With the exponentially expanding user base and item catalog in contemporary e-commerce, this design is admittedly becoming memory-inefficient. To facilitate lightweight recommendation, reinforcement learning (RL) has recently opened up opportunities for identifying varying embedding sizes for different users/items. However, challenged by search efficiency and learning an optimal RL policy, existing RL-based methods are restricted to highly discrete, predefined embedding size choices. This leads to a largely overlooked potential of introducing finer granularity into embedding sizes to obtain better recommendation effectiveness under a given memory budget. In this paper, we propose continuous input embedding size search (CIESS), a novel RL-based method that operates on a continuous search space with arbitrary embedding sizes to choose from. In CIESS, we further present an innovative random walk-based exploration strategy to allow the RL policy to efficiently explore more candidate embedding sizes and converge to a better decision. CIESS is also model-agnostic and hence generalizable to a variety of latent factor RSs, whilst experiments on two real-world datasets have shown state-of-the-art performance of CIESS under different memory budgets when paired with three popular recommendation models.

Related papers

• MALLOC: Benchmarking the Memory-aware Long Sequence Compression for Large Sequential Recommendation [84.53415999381203]
  MALLOC is a benchmark for memory-aware long sequence compression.<n>It is integrated into state-of-the-art recommenders, enabling a reproducible and evaluation platform.
  arXiv  Detail & Related papers  (2026-01-28T04:11:50Z)
• Large Memory Network for Recommendation [21.618829330517844]
  Large Memory Network (LMN) is a novel idea by compressing and storing user history behavior information in a large-scale memory block.<n>LMN has been fully deployed in Douyin E-Commerce Search (ECS), serving millions of users each day.
  arXiv  Detail & Related papers  (2025-02-08T13:08:11Z)
• Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs [4.165917157093442]
  This paper introduces a novel Scalable Cross-Entropy (SCE) loss function in the sequential learning setup.<n>It approximates the CE loss for datasets with large-size catalogs, enhancing both time efficiency and memory usage without compromising recommendations quality.<n> Experimental results on multiple datasets demonstrate the effectiveness of SCE in reducing peak memory usage by a factor of up to 100 compared to the alternatives.
  arXiv  Detail & Related papers  (2024-09-27T13:17:59Z)
• Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation [50.837277466987345]
  We focus on the field of large language models (LLMs) for recommendation. We propose RecLoRA, which incorporates a Personalized LoRA module that maintains independent LoRAs for different users. We also design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces.
  arXiv  Detail & Related papers  (2024-08-07T04:20:28Z)
• Scalable Dynamic Embedding Size Search for Streaming Recommendation [47.69328280479569]
  Real-world recommender systems often operate in streaming recommendation scenarios.<n>Number of users and items continues to grow, leading to substantial storage resource consumption.<n>We learn Lightweight Embeddings for streaming recommendation, called SCALL, which can adaptively adjust the embedding sizes of users/items.
  arXiv  Detail & Related papers  (2024-07-22T06:37:24Z)
• A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems [67.52782366565658]
  State-of-the-art recommender systems (RSs) depend on categorical features, which ecoded by embedding vectors, resulting in excessively large embedding tables.<n>Despite the prosperity of lightweight embedding-based RSs, a wide diversity is seen in evaluation protocols.<n>This study investigates various LERS' performance, efficiency, and cross-task transferability via a thorough benchmarking process.
  arXiv  Detail & Related papers  (2024-06-25T07:45:00Z)
• Budgeted Embedding Table For Recommender Systems [39.882452493920724]
  Budgeted Embedding Table (BET) is a novel method that generates table-level actions that are guaranteed to meet memory budgets.<n>BET is paired with three popular recommender models under different memory budgets.
  arXiv  Detail & Related papers  (2023-10-23T12:53:22Z)
• Reinforcement Learning from Diverse Human Preferences [68.4294547285359]
  This paper develops a method for crowd-sourcing preference labels and learning from diverse human preferences. The proposed method is tested on a variety of tasks in DMcontrol and Meta-world. It has shown consistent and significant improvements over existing preference-based RL algorithms when learning from diverse feedback.
  arXiv  Detail & Related papers  (2023-01-27T15:18:54Z)
• Differentiable Neural Input Search for Recommender Systems [26.88124270897381]
  Differentiable Neural Input Search (DNIS) is a method that searches for mixed feature embedding dimensions in a more flexible space. DNIS is model-agnostic and can be seamlessly incorporated with existing latent factor models for recommendation.
  arXiv  Detail & Related papers  (2020-06-08T10:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.