Light distillation for Incremental Graph Convolution Collaborative Filtering

• URL: http://arxiv.org/abs/2505.19810v1
• Date: Mon, 26 May 2025 10:47:26 GMT
• Title: Light distillation for Incremental Graph Convolution Collaborative Filtering
• Abstract summary: We propose a light preference-driven distillation method to distill the preference score of a user for an item directly from historical interactions.<n>The proposed method can save training time from 1.5x to 9.5x compared to the existing methods and improves Recall@20 by 5.41% and 10.64% from the fine-tune method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recommender systems presently utilize vast amounts of data and play a pivotal role in enhancing user experiences. Graph Convolution Networks (GCNs) have surfaced as highly efficient models within the realm of recommender systems due to their ability to capture extensive relational information. The continuously expanding volume of data may render the training of GCNs excessively costly. To tackle this problem, incrementally training GCNs as new data blocks come in has become a vital research direction. Knowledge distillation techniques have been explored as a general paradigm to train GCNs incrementally and alleviate the catastrophic forgetting problem that typically occurs in incremental settings. However, we argue that current methods based on knowledge distillation introduce additional parameters and have a high model complexity, which results in unrealistic training time consumption in an incremental setting and thus difficult to actually deploy in the real world. In this work, we propose a light preference-driven distillation method to distill the preference score of a user for an item directly from historical interactions, which reduces the training time consumption in the incremental setting significantly without noticeable loss in performance. The experimental result on two general datasets shows that the proposed method can save training time from 1.5x to 9.5x compared to the existing methods and improves Recall@20 by 5.41% and 10.64% from the fine-tune method.

Related papers

• Efficient Cross-Architecture Knowledge Transfer for Large-Scale Online User Response Prediction [43.353811451921466]
  CrossAdapt is a two-stage framework for efficient cross-architecture knowledge transfer.<n>The offline stage enables rapid embedding transfer via dimension-adaptive projections without iterative training.<n>The online stage introduces asymmetric co-distillation, where students update frequently while teachers update infrequently.
  arXiv  Detail & Related papers  (2026-02-02T08:00:36Z)
• Data-Efficient RLVR via Off-Policy Influence Guidance [84.60336960383867]
  This work proposes a theoretically-grounded approach using influence functions to estimate the contribution of each data point to the learning objective.<n>We develop textbfCurriculum textbfRL with textbfOff-textbfPolicy textInfluence guidance (textbfCROPI), a multi-stage RL framework that iteratively selects the most influential data for the current policy.
  arXiv  Detail & Related papers  (2025-10-30T13:40:52Z)
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward [54.708851958671794]
  We propose a Data-Efficient Policy Optimization pipeline that combines optimized strategies for both offline and online data selection.<n>In offline phase, we curate a high-quality subset of training samples based on diversity, influence, and appropriate difficulty.<n>During online RLVR training, we introduce a sample-level explorability metric to dynamically filter samples with low exploration potential.
  arXiv  Detail & Related papers  (2025-09-01T10:04:20Z)
• Synthetic Data is Sufficient for Zero-Shot Visual Generalization from Offline Data [22.840912154067325]
  policies trained on offline data often struggle to generalise due to limited exposure to diverse states.<n>This makes it challenging to leverage vision-based offline data in training robust agents that can generalize to unseen environments.<n>We propose a two-step process, first augmenting the originally collected offline data to improve zero-shot generalization by introducing diversity, then using a diffusion model to generate additional data in latent space.
  arXiv  Detail & Related papers  (2025-08-17T13:01:15Z)
• PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning [17.230781041043823]
  We propose a novel prompt-based method for online continual learning (OCL) that includes 4 main components.<n>Our proposed method achieves significantly higher performance than the current SOTAs in CIFAR100, ImageNet-R, ImageNet-A, and CUB dataset.
  arXiv  Detail & Related papers  (2025-07-16T15:04:46Z)
• SPEQ: Offline Stabilization Phases for Efficient Q-Learning in High Update-To-Data Ratio Reinforcement Learning [51.10866035483686]
  High update-to-data (UTD) ratio algorithms in reinforcement learning (RL) improve sample efficiency but incur high computational costs, limiting real-world scalability.<n>We propose Offline Stabilization Phases for Efficient Q-Learning (SPEQ), an RL algorithm that combines low-UTD online training with periodic offline stabilization phases.<n>During these phases, Q-functions are fine-tuned with high UTD ratios on a fixed replay buffer, reducing redundant updates on suboptimal data.
  arXiv  Detail & Related papers  (2025-01-15T09:04:19Z)
• PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection [65.24854366973794]
  Node-level graph anomaly detection (GAD) plays a critical role in identifying anomalous nodes from graph-structured data in domains such as medicine, social networks, and e-commerce. We introduce a simple method termed PREprocessing and Matching (PREM for short) to improve the efficiency of GAD. Our approach streamlines GAD, reducing time and memory consumption while maintaining powerful anomaly detection capabilities.
  arXiv  Detail & Related papers  (2023-10-18T02:59:57Z)
• AST: Effective Dataset Distillation through Alignment with Smooth and High-Quality Expert Trajectories [18.266786462036553]
  We propose an effective DD framework named AST, standing for Alignment with Smooth and high-quality expert Trajectories. We conduct extensive experiments on datasets of different scales, sizes, and resolutions.
  arXiv  Detail & Related papers  (2023-10-16T16:13:53Z)
• Incorporating Supervised Domain Generalization into Data Augmentation [4.14360329494344]
  We propose a method, contrastive semantic alignment(CSA) loss, to improve robustness and training efficiency of data augmentation. Experiments on the CIFAR-100 and CUB datasets show that the proposed method improves the robustness and training efficiency of typical data augmentations.
  arXiv  Detail & Related papers  (2023-10-02T09:20:12Z)
• Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay [52.251188477192336]
  Few-shot class-incremental learning (FSCIL) has been proposed aiming to enable a deep learning system to incrementally learn new classes with limited data. We show through empirical results that adopting the data replay is surprisingly favorable. We propose using data-free replay that can synthesize data by a generator without accessing real data.
  arXiv  Detail & Related papers  (2022-07-22T17:30:51Z)
• Online Convolutional Re-parameterization [51.97831675242173]
  We present online convolutional re- parameterization (OREPA), a two-stage pipeline, aiming to reduce the huge training overhead by squeezing the complex training-time block into a single convolution. Compared with the state-of-the-art re-param models, OREPA is able to save the training-time memory cost by about 70% and accelerate the training speed by around 2x. We also conduct experiments on object detection and semantic segmentation and show consistent improvements on the downstream tasks.
  arXiv  Detail & Related papers  (2022-04-02T09:50:19Z)
• Causal Incremental Graph Convolution for Recommender System Retraining [89.25922726558875]
  Real-world recommender system needs to be regularly retrained to keep with the new data. In this work, we consider how to efficiently retrain graph convolution network (GCN) based recommender models.
  arXiv  Detail & Related papers  (2021-08-16T04:20:09Z)
• Spirit Distillation: Precise Real-time Prediction with Insufficient Data [4.6247655021017655]
  We propose a new training framework named Spirit Distillation(SD) It extends the ideas of fine-tuning-based transfer learning(FTT) and feature-based knowledge distillation. Results demonstrate the boosting performance in segmentation(mIOU) and high-precision accuracy boost by 1.4% and 8.2% respectively.
  arXiv  Detail & Related papers  (2021-03-25T10:23:30Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)
• Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
  We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
  arXiv  Detail & Related papers  (2020-06-10T08:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.