DNS-Rec: Data-aware Neural Architecture Search for Recommender Systems

• URL: http://arxiv.org/abs/2402.00390v2
• Date: Thu, 19 Dec 2024 14:28:19 GMT
• Title: DNS-Rec: Data-aware Neural Architecture Search for Recommender Systems
• Authors: Sheng Zhang, Maolin Wang, Yao Zhao, Chenyi Zhuang, Jinjie Gu, Ruocheng Guo, Xiangyu Zhao, Zijian Zhang, Hongzhi Yin,
• Abstract summary: This paper addresses the computational overhead and resource inefficiency prevalent in Sequential Recommender Systems (SRSs)<n>We introduce an innovative approach combining pruning methods with advanced model designs.<n>Our principal contribution is the development of a Data-aware Neural Architecture Search for Recommender System (DNS-Rec)
• Score: 79.76519917171261
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the era of data proliferation, efficiently sifting through vast information to extract meaningful insights has become increasingly crucial. This paper addresses the computational overhead and resource inefficiency prevalent in existing Sequential Recommender Systems (SRSs). We introduce an innovative approach combining pruning methods with advanced model designs. Furthermore, we delve into resource-constrained Neural Architecture Search (NAS), an emerging technique in recommender systems, to optimize models in terms of FLOPs, latency, and energy consumption while maintaining or enhancing accuracy. Our principal contribution is the development of a Data-aware Neural Architecture Search for Recommender System (DNS-Rec). DNS-Rec is specifically designed to tailor compact network architectures for attention-based SRS models, thereby ensuring accuracy retention. It incorporates data-aware gates to enhance the performance of the recommendation network by learning information from historical user-item interactions. Moreover, DNS-Rec employs a dynamic resource constraint strategy, stabilizing the search process and yielding more suitable architectural solutions. We demonstrate the effectiveness of our approach through rigorous experiments conducted on three benchmark datasets, which highlight the superiority of DNS-Rec in SRSs. Our findings set a new standard for future research in efficient and accurate recommendation systems, marking a significant step forward in this rapidly evolving field.

Related papers

• Meta knowledge assisted Evolutionary Neural Architecture Search [38.55611683982936]
  This paper introduces an efficient EC-based NAS method to solve problems via an innovative meta-learning framework. An adaptive surrogate model is designed through an adaptive threshold to select the potential architectures. Experiments on CIFAR-10, CIFAR-100, and ImageNet1K datasets demonstrate that the proposed method achieves high performance comparable to that of many state-of-the-art peer methods.
  arXiv  Detail & Related papers  (2025-04-30T11:43:07Z)
• Behavior Importance-Aware Graph Neural Architecture Search for Cross-Domain Recommendation [49.03831702224862]
  Cross-domain recommendation (CDR) mitigates data sparsity and cold-start issues in recommendation systems. Recent CDR approaches using graph neural networks (GNNs) capture complex user-item interactions. BiGNAS is a framework that jointly optimize GNN architecture and data importance for CDR.
  arXiv  Detail & Related papers  (2025-03-11T04:30:18Z)
• Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models [26.353428245346166]
  The Extract-Refine-Retrieve-Read (ERRR) framework is designed to bridge the pre-retrieval information gap in Retrieval-Augmented Generation (RAG) systems. Unlike conventional query optimization techniques used in RAG, the ERRR framework begins by extracting knowledge from Large Language Models (LLMs)
  arXiv  Detail & Related papers  (2024-11-12T14:12:45Z)
• Exploring Applications of State Space Models and Advanced Training Techniques in Sequential Recommendations: A Comparative Study on Efficiency and Performance [41.677784966514686]
  This research focuses on three promising directions in sequential recommendations. The first is to enhance speed through the use of State Space Models (SSM), as they can achieve SOTA results in the sequential recommendations domain with lower latency, memory, and inference costs. The second is to improve the quality of recommendations with Large Language Models (LLMs), via Monolithic Preference Optimization without Reference Model (ORPO), and implementing adaptive batch- and step-size algorithms to reduce costs and accelerate training processes.
  arXiv  Detail & Related papers  (2024-08-10T18:09:10Z)
• A Study on the Implementation Method of an Agent-Based Advanced RAG System Using Graph [0.0]
  This study implements an advanced RAG system based on Graph technology to develop high-quality generative AI services. It employs LangGraph to evaluate the reliability of retrieved information and synthesizes diverse data to generate more accurate and enhanced responses.
  arXiv  Detail & Related papers  (2024-07-29T13:26:43Z)
• A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems [67.52782366565658]
  State-of-the-art recommender systems (RSs) depend on categorical features, which ecoded by embedding vectors, resulting in excessively large embedding tables. Despite the prosperity of lightweight embedding-based RSs, a wide diversity is seen in evaluation protocols. This study investigates various LERS' performance, efficiency, and cross-task transferability via a thorough benchmarking process.
  arXiv  Detail & Related papers  (2024-06-25T07:45:00Z)
• Dataset Regeneration for Sequential Recommendation [69.93516846106701]
  We propose a data-centric paradigm for developing an ideal training dataset using a model-agnostic dataset regeneration framework called DR4SR. To demonstrate the effectiveness of the data-centric paradigm, we integrate our framework with various model-centric methods and observe significant performance improvements across four widely adopted datasets.
  arXiv  Detail & Related papers  (2024-05-28T03:45:34Z)
• Efficient Architecture Search via Bi-level Data Pruning [70.29970746807882]
  This work pioneers an exploration into the critical role of dataset characteristics for DARTS bi-level optimization. We introduce a new progressive data pruning strategy that utilizes supernet prediction dynamics as the metric. Comprehensive evaluations on the NAS-Bench-201 search space, DARTS search space, and MobileNet-like search space validate that BDP reduces search costs by over 50%.
  arXiv  Detail & Related papers  (2023-12-21T02:48:44Z)
• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [64.04617968947697]
  We introduce a novel data-model co-design perspective: to promote superior weight sparsity. Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
  arXiv  Detail & Related papers  (2023-12-03T13:50:24Z)
• Embedding in Recommender Systems: A Survey [67.67966158305603]
  A crucial aspect is embedding techniques that covert the high-dimensional discrete features, such as user and item IDs, into low-dimensional continuous vectors. Applying embedding techniques captures complex entity relationships and has spurred substantial research. This survey covers embedding methods like collaborative filtering, self-supervised learning, and graph-based techniques.
  arXiv  Detail & Related papers  (2023-10-28T06:31:06Z)
• Modeling Time-Series and Spatial Data for Recommendations and Other Applications [1.713291434132985]
  We address the problems that may arise due to the poor quality of CTES data being fed into a recommender system. To improve the quality of the CTES data, we address a fundamental problem of overcoming missing events in temporal sequences. We extend their abilities to design solutions for large-scale CTES retrieval and human activity prediction.
  arXiv  Detail & Related papers  (2022-12-25T09:34:15Z)
• Re-visiting Reservoir Computing architectures optimized by Evolutionary Algorithms [0.0]
  Evolutionary Algorithms (EAs) have been applied to improve Neural Networks (NNs) architectures. We provide a systematic brief survey about applications of EAs on the specific domain of the recurrent NNs named Reservoir Computing (RC)
  arXiv  Detail & Related papers  (2022-11-11T14:50:54Z)
• NASRec: Weight Sharing Neural Architecture Search for Recommender Systems [40.54254555949057]
  We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. Our results on three Click-Through Rates (CTR) prediction benchmarks show that NASRec can outperform both manually designed models and existing NAS methods.
  arXiv  Detail & Related papers  (2022-07-14T20:15:11Z)
• A Hybrid Framework for Sequential Data Prediction with End-to-End Optimization [0.0]
  We investigate nonlinear prediction in an online setting and introduce a hybrid model that effectively mitigates hand-designed features and manual model selection issues. We employ a recurrent neural network (LSTM) for adaptive feature extraction from sequential data and a gradient boosting machinery (soft GBDT) for effective supervised regression. We demonstrate the learning behavior of our algorithm on synthetic data and the significant performance improvements over the conventional methods over various real life datasets.
  arXiv  Detail & Related papers  (2022-03-25T17:13:08Z)
• Learning Where To Look -- Generative NAS is Surprisingly Efficient [11.83842808044211]
  We propose a generative model, paired with a surrogate predictor, that iteratively learns to generate samples from increasingly promising latent subspaces. This approach leads to very effective and efficient architecture search, while keeping the query amount low.
  arXiv  Detail & Related papers  (2022-03-16T16:27:11Z)
• LoRD-Net: Unfolded Deep Detection Network with Low-Resolution Receivers [104.01415343139901]
  We propose a deep detector entitled LoRD-Net for recovering information symbols from one-bit measurements. LoRD-Net has a task-based architecture dedicated to recovering the underlying signal of interest. We evaluate the proposed receiver architecture for one-bit signal recovery in wireless communications.
  arXiv  Detail & Related papers  (2021-02-05T04:26:05Z)
• Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search [50.40004966087121]
  We introduce a new reinforcement learning based neural architecture search (NAS) methodology for generative adversarial network (GAN) architecture search. The key idea is to formulate the GAN architecture search problem as a Markov decision process (MDP) for smoother architecture sampling. We exploit an off-policy GAN architecture search algorithm that makes efficient use of the samples generated by previous policies.
  arXiv  Detail & Related papers  (2020-07-17T18:29:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.