EASRec: Elastic Architecture Search for Efficient Long-term Sequential
Recommender Systems
- URL: http://arxiv.org/abs/2402.00390v1
- Date: Thu, 1 Feb 2024 07:22:52 GMT
- Title: EASRec: Elastic Architecture Search for Efficient Long-term Sequential
Recommender Systems
- Authors: Sheng Zhang, Maolin Wang, Yao Zhao, Chenyi Zhuang, Jinjie Gu, Ruocheng
Guo, Xiangyu Zhao, Zijian Zhang, Hongzhi Yin
- Abstract summary: Current Sequential Recommender Systems (SRSs) suffer from computational and resource inefficiencies.
We develop the Elastic Architecture Search for Efficient Long-term Sequential Recommender Systems (EASRec)
EASRec introduces data-aware gates that leverage historical information from input data batch to improve the performance of the recommendation network.
- Score: 82.76483989905961
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this age where data is abundant, the ability to distill meaningful
insights from the sea of information is essential. Our research addresses the
computational and resource inefficiencies that current Sequential Recommender
Systems (SRSs) suffer from. especially those employing attention-based models
like SASRec, These systems are designed for next-item recommendations in
various applications, from e-commerce to social networks. However, such systems
suffer from substantial computational costs and resource consumption during the
inference stage. To tackle these issues, our research proposes a novel method
that combines automatic pruning techniques with advanced model architectures.
We also explore the potential of resource-constrained Neural Architecture
Search (NAS), a technique prevalent in the realm of recommendation systems, to
fine-tune models for reduced FLOPs, latency, and energy usage while retaining
or even enhancing accuracy. The main contribution of our work is developing the
Elastic Architecture Search for Efficient Long-term Sequential Recommender
Systems (EASRec). This approach aims to find optimal compact architectures for
attention-based SRSs, ensuring accuracy retention. EASRec introduces data-aware
gates that leverage historical information from input data batch to improve the
performance of the recommendation network. Additionally, it utilizes a dynamic
resource constraint approach, which standardizes the search process and results
in more appropriate architectures. The effectiveness of our methodology is
validated through exhaustive experiments on three benchmark datasets, which
demonstrates EASRec's superiority in SRSs. Our research set a new standard for
future exploration into efficient and accurate recommender systems, signifying
a substantial advancement within this swiftly advancing field.
Related papers
- Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models [26.353428245346166]
The Extract-Refine-Retrieve-Read (ERRR) framework is designed to bridge the pre-retrieval information gap in Retrieval-Augmented Generation (RAG) systems.
Unlike conventional query optimization techniques used in RAG, the ERRR framework begins by extracting knowledge from Large Language Models (LLMs)
arXiv Detail & Related papers (2024-11-12T14:12:45Z) - Dataset Regeneration for Sequential Recommendation [69.93516846106701]
We propose a data-centric paradigm for developing an ideal training dataset using a model-agnostic dataset regeneration framework called DR4SR.
To demonstrate the effectiveness of the data-centric paradigm, we integrate our framework with various model-centric methods and observe significant performance improvements across four widely adopted datasets.
arXiv Detail & Related papers (2024-05-28T03:45:34Z) - Efficient Architecture Search via Bi-level Data Pruning [70.29970746807882]
This work pioneers an exploration into the critical role of dataset characteristics for DARTS bi-level optimization.
We introduce a new progressive data pruning strategy that utilizes supernet prediction dynamics as the metric.
Comprehensive evaluations on the NAS-Bench-201 search space, DARTS search space, and MobileNet-like search space validate that BDP reduces search costs by over 50%.
arXiv Detail & Related papers (2023-12-21T02:48:44Z) - Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [64.04617968947697]
We introduce a novel data-model co-design perspective: to promote superior weight sparsity.
Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
arXiv Detail & Related papers (2023-12-03T13:50:24Z) - Re-visiting Reservoir Computing architectures optimized by Evolutionary
Algorithms [0.0]
Evolutionary Algorithms (EAs) have been applied to improve Neural Networks (NNs) architectures.
We provide a systematic brief survey about applications of EAs on the specific domain of the recurrent NNs named Reservoir Computing (RC)
arXiv Detail & Related papers (2022-11-11T14:50:54Z) - NASRec: Weight Sharing Neural Architecture Search for Recommender
Systems [40.54254555949057]
We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing.
Our results on three Click-Through Rates (CTR) prediction benchmarks show that NASRec can outperform both manually designed models and existing NAS methods.
arXiv Detail & Related papers (2022-07-14T20:15:11Z) - A Hybrid Framework for Sequential Data Prediction with End-to-End
Optimization [0.0]
We investigate nonlinear prediction in an online setting and introduce a hybrid model that effectively mitigates hand-designed features and manual model selection issues.
We employ a recurrent neural network (LSTM) for adaptive feature extraction from sequential data and a gradient boosting machinery (soft GBDT) for effective supervised regression.
We demonstrate the learning behavior of our algorithm on synthetic data and the significant performance improvements over the conventional methods over various real life datasets.
arXiv Detail & Related papers (2022-03-25T17:13:08Z) - A novel Deep Neural Network architecture for non-linear system
identification [78.69776924618505]
We present a novel Deep Neural Network (DNN) architecture for non-linear system identification.
Inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function)
This architecture allows for automatic complexity selection based solely on available data.
arXiv Detail & Related papers (2021-06-06T10:06:07Z) - LoRD-Net: Unfolded Deep Detection Network with Low-Resolution Receivers [104.01415343139901]
We propose a deep detector entitled LoRD-Net for recovering information symbols from one-bit measurements.
LoRD-Net has a task-based architecture dedicated to recovering the underlying signal of interest.
We evaluate the proposed receiver architecture for one-bit signal recovery in wireless communications.
arXiv Detail & Related papers (2021-02-05T04:26:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.