Learning from the Future: Privileged Self-Distillation for Sequential Recommendation

• URL: http://arxiv.org/abs/2607.27055v1
• Date: Wed, 29 Jul 2026 15:47:22 GMT
• Title: Learning from the Future: Privileged Self-Distillation for Sequential Recommendation
• Abstract summary: We propose a framework that separates learning-time information from inference-time information.<n> Privileged Self-Distillation (PSD) separates learning-time information from inference-time information.<n>PSD is optimized end-to-end in a single stage, leaving the deployed model and inference cost unchanged.
• Score: 51.993699802866956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential recommenders are commonly trained with one-hot next-item labels under a causal (prefix-only) objective aligned with inference. While deployment-compatible, this supervision offers little insight into relative preferences among non-target items. Yet logged interaction sequences contain an additional supervisory source: interactions following the target often reveal how user intent evolves, making the target easier to interpret. We treat these future interactions as training-only privileged information, available during learning but not at inference. This raises a natural question: can future interactions provide richer supervision while keeping training aligned with inference-time prediction? We propose Privileged Self-Distillation (PSD), a framework that separates learning-time information from inference-time information. PSD applies two attention masks to the same backbone: a future-aware view yields a privileged teacher distribution conditioned on past and future interactions, while a prefix-only view yields the student distribution used for deployment. Distilling the privileged distribution converts future interactions into training-only supervision rather than inference-time inputs. Since both views share a backbone, the teacher's advantage is purely informational, not architectural, removing the need for a separately pretrained teacher and letting its supervision adapt as the student evolves. PSD further uses an advantage-reachability gate to focus distillation on teacher signals likely supported by the observed prefix, along with a momentum-averaged teacher for stable targets. The framework is optimized end-to-end in a single stage, leaving the deployed model and inference cost unchanged. Experiments across public benchmarks and diverse backbones show consistent improvements.

Related papers

• Learning to Track from Privileged Target Appearances [67.80976117600649]
  We quantify a bottleneck with a non-deployable oracle that supplies an exact current-frame target crop.<n>We introduce Privileged Appearance Transfer for Tracking (PATT), a teacher-student training framework that transfers these privileged appearances to a deployable tracker.<n>PATT achieves consistent gains under both long- and short-term tracking protocols.
  arXiv  Detail & Related papers  (2026-09-02T11:43:43Z)
• Rank-Then-Act: Reward-Free Control from Frame-Order Progress [5.357451930622806]
  Rank-Then-Act (RTA) is a framework for learning control policies from expert video demonstrations without environment rewards.<n>We train a Vision-Language Model (VLM) offline as a progress-based ordinal scorer.<n>We compute the Spearman rank correlation between predicted progress rankings and true temporal indices, yielding a bounded, scale-invariant learning signal.
  arXiv  Detail & Related papers  (2026-07-02T08:50:32Z)
• Privileged Foresight Distillation: Zero-Cost Future Correction for World Action Models [1.5821415295352772]
  We show that joint training induces an action-conditioned correction that privileged future observations impose on action denoising.<n>We introduce emphPrivileged Foresight Distillation (PFD), which transfers this residual from a training-time teacher into a small adapter on a current-only student.
  arXiv  Detail & Related papers  (2026-04-28T16:58:52Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• Self-Supervised Learning as Discrete Communication [3.33876524834826]
  Most self-supervised learning (SSL) methods learn continuous visual representations by aligning different views of the same input.<n>We frame visual self-supervised learning as a discrete communication process between a teacher and a student network.<n>We show that periodically reinitializing the projection head strengthens this effect by encouraging embeddings that remain predictive.
  arXiv  Detail & Related papers  (2026-02-10T13:24:06Z)
• Bridging Past and Future: Distribution-Aware Alignment for Time Series Forecasting [30.686607555300366]
  We introduce TimeAlign, a representation-learning framework for time series forecasters.<n>We explicitly align past and future representations, thereby bridging the distributional gap between input histories and future targets.<n>The gains arise primarily from correcting frequency mismatches between historical inputs and future outputs.
  arXiv  Detail & Related papers  (2025-09-17T17:12:39Z)
• Beyond Prototypes: Semantic Anchor Regularization for Better Representation Learning [82.29761875805369]
  One of the ultimate goals of representation learning is to achieve compactness within a class and well-separability between classes. We propose a novel perspective to use pre-defined class anchors serving as feature centroid to unidirectionally guide feature learning. The proposed Semantic Anchor Regularization (SAR) can be used in a plug-and-play manner in the existing models.
  arXiv  Detail & Related papers  (2023-12-19T05:52:38Z)
• Learning from Future: A Novel Self-Training Framework for Semantic Segmentation [33.66516999361252]
  Self-training has shown great potential in semi-supervised learning. We propose a novel self-training strategy, which allows the model to learn from the future. We experimentally demonstrate the effectiveness and superiority of our approach under a wide range of settings.
  arXiv  Detail & Related papers  (2022-09-15T01:39:46Z)
• Self-Regulated Learning for Egocentric Video Activity Anticipation [147.9783215348252]
  Self-Regulated Learning (SRL) aims to regulate the intermediate representation consecutively to produce representation that emphasizes the novel information in the frame of the current time-stamp. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets.
  arXiv  Detail & Related papers  (2021-11-23T03:29:18Z)
• Privileged Knowledge Distillation for Online Action Detection [114.5213840651675]
  Online Action Detection (OAD) in videos is proposed as a per-frame labeling task to address the real-time prediction tasks. This paper presents a novel learning-with-privileged based framework for online action detection where the future frames only observable at the training stages are considered as a form of privileged information.
  arXiv  Detail & Related papers  (2020-11-18T08:52:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.