Retrieval and Distill: A Temporal Data Shift-Free Paradigm for Online Recommendation System

• URL: http://arxiv.org/abs/2404.15678v4
• Date: Thu, 13 Jun 2024 07:53:06 GMT
• Title: Retrieval and Distill: A Temporal Data Shift-Free Paradigm for Online Recommendation System
• Authors: Lei Zheng, Ning Li, Weinan Zhang, Yong Yu,
• Abstract summary: Current recommendation systems are significantly affected by a serious issue of temporal data shift. Most existing models focus on utilizing updated data, overlooking the transferable, temporal data shift-free information that can be learned from shifting data. We propose a retrieval-based recommendation system framework that can train a data shift-free relevance network using shifting data.
• Score: 31.594407236146186
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current recommendation systems are significantly affected by a serious issue of temporal data shift, which is the inconsistency between the distribution of historical data and that of online data. Most existing models focus on utilizing updated data, overlooking the transferable, temporal data shift-free information that can be learned from shifting data. We propose the Temporal Invariance of Association theorem, which suggests that given a fixed search space, the relationship between the data and the data in the search space keeps invariant over time. Leveraging this principle, we designed a retrieval-based recommendation system framework that can train a data shift-free relevance network using shifting data, significantly enhancing the predictive performance of the original model in the recommendation system. However, retrieval-based recommendation models face substantial inference time costs when deployed online. To address this, we further designed a distill framework that can distill information from the relevance network into a parameterized module using shifting data. The distilled model can be deployed online alongside the original model, with only a minimal increase in inference time. Extensive experiments on multiple real datasets demonstrate that our framework significantly improves the performance of the original model by utilizing shifting data.

Related papers

• Federated Dynamic Modeling and Learning for Spatiotemporal Data Forecasting [0.8568432695376288]
  This paper presents an advanced Federated Learning (FL) framework for forecasting complextemporal data, improving upon recent state-of-the-art models. The resulting architecture significantly improves the model's capacity to handle complex temporal patterns in diverse forecasting applications. The efficiency of our approach is demonstrated through extensive experiments on real-world applications, including public datasets for multimodal transport demand forecasting and private datasets for Origin-Destination (OD) matrix forecasting in urban areas.
  arXiv  Detail & Related papers  (2025-03-06T15:16:57Z)
• Tackling Data Heterogeneity in Federated Time Series Forecasting [61.021413959988216]
  Time series forecasting plays a critical role in various real-world applications, including energy consumption prediction, disease transmission monitoring, and weather forecasting. Most existing methods rely on a centralized training paradigm, where large amounts of data are collected from distributed devices to a central cloud server. We propose a novel framework, Fed-TREND, to address data heterogeneity by generating informative synthetic data as auxiliary knowledge carriers.
  arXiv  Detail & Related papers  (2024-11-24T04:56:45Z)
• A Scalable Approach to Covariate and Concept Drift Management via Adaptive Data Segmentation [0.562479170374811]
  In many real-world applications, continuous machine learning (ML) systems are crucial but prone to data drift. Traditional drift adaptation methods typically update models using ensemble techniques, often discarding drifted historical data. We contend that explicitly incorporating drifted data into the model training process significantly enhances model accuracy and robustness.
  arXiv  Detail & Related papers  (2024-11-23T17:35:23Z)
• TimeSieve: Extracting Temporal Dynamics through Information Bottlenecks [31.10683149519954]
  We propose an innovative time series forecasting model TimeSieve. Our approach employs wavelet transforms to preprocess time series data, effectively capturing multi-scale features. Our results validate the effectiveness of our approach in addressing the key challenges in time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T15:58:12Z)
• Dataset Regeneration for Sequential Recommendation [69.93516846106701]
  We propose a data-centric paradigm for developing an ideal training dataset using a model-agnostic dataset regeneration framework called DR4SR. To demonstrate the effectiveness of the data-centric paradigm, we integrate our framework with various model-centric methods and observe significant performance improvements across four widely adopted datasets.
  arXiv  Detail & Related papers  (2024-05-28T03:45:34Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Quilt: Robust Data Segment Selection against Concept Drifts [30.62320149405819]
  Continuous machine learning pipelines are common in industrial settings where models are periodically trained on data streams. concept drifts may occur in data streams where the joint distribution of the data X and label y, P(X, y), changes over time and possibly degrade model accuracy. Existing concept drift adaptation approaches mostly focus on updating the model to the new data and tend to discard the drifted historical data. We propose Quilt, a data-centric framework for identifying and selecting data segments that maximize model accuracy.
  arXiv  Detail & Related papers  (2023-12-15T11:10:34Z)
• Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System [46.39662315849883]
  We introduce CLMFormer, a novel framework that mitigates redundancy through curriculum learning and a memory-driven decoder. CLMFormer consistently improves Transformer-based models by up to 30%, demonstrating its effectiveness in long-horizon forecasting.
  arXiv  Detail & Related papers  (2022-07-16T04:05:15Z)
• Employing chunk size adaptation to overcome concept drift [2.277447144331876]
  We propose a new Chunk Adaptive Restoration framework that can be adapted to any block-based data stream classification algorithm. The proposed algorithm adjusts the data chunk size in the case of concept drift detection to minimize the impact of the change on the predictive performance of the used model.
  arXiv  Detail & Related papers  (2021-10-25T12:36:22Z)
• Real-time Drift Detection on Time-series Data [0.6303112417588329]
  We propose an approach called Unsupervised Temporal Drift Detector or UTDD to account flexibly for seasonal variation. This approach efficiently detects temporal concept drift in time series data in the absence of ground truth.
  arXiv  Detail & Related papers  (2021-10-12T22:09:29Z)
• Time-Series Imputation with Wasserstein Interpolation for Optimal Look-Ahead-Bias and Variance Tradeoff [66.59869239999459]
  In finance, imputation of missing returns may be applied prior to training a portfolio optimization model. There is an inherent trade-off between the look-ahead-bias of using the full data set for imputation and the larger variance in the imputation from using only the training data. We propose a Bayesian posterior consensus distribution which optimally controls the variance and look-ahead-bias trade-off in the imputation.
  arXiv  Detail & Related papers  (2021-02-25T09:05:35Z)
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization [104.87483578308526]
  We propose the model S3-Rec, which stands for Self-Supervised learning for Sequential Recommendation. For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence. Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-18T11:44:10Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.