Abacus: Self-Supervised Event Counting-Aligned Distributional Pretraining for Sequential User Modeling

• URL: http://arxiv.org/abs/2512.16581v1
• Date: Thu, 18 Dec 2025 14:24:51 GMT
• Title: Abacus: Self-Supervised Event Counting-Aligned Distributional Pretraining for Sequential User Modeling
• Abstract summary: We introduce Abacus, a novel approach of predicting the empirical frequency distribution of user events.<n>Abacus pretraining outperforms existing methods accelerating downstream task convergence.
• Score: 1.3416169841532526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling user purchase behavior is a critical challenge in display advertising systems, necessary for real-time bidding. The difficulty arises from the sparsity of positive user events and the stochasticity of user actions, leading to severe class imbalance and irregular event timing. Predictive systems usually rely on hand-crafted "counter" features, overlooking the fine-grained temporal evolution of user intent. Meanwhile, current sequential models extract direct sequential signal, missing useful event-counting statistics. We enhance deep sequential models with self-supervised pretraining strategies for display advertising. Especially, we introduce Abacus, a novel approach of predicting the empirical frequency distribution of user events. We further propose a hybrid objective unifying Abacus with sequential learning objectives, combining stability of aggregated statistics with the sequence modeling sensitivity. Experiments on two real-world datasets show that Abacus pretraining outperforms existing methods accelerating downstream task convergence, while hybrid approach yields up to +6.1% AUC compared to the baselines.

Related papers

• Diffusion Models for Adaptive Sequential Data Generation [18.424873249038004]
  Time-indexed observations are used for prediction, simulation, risk assessment, and data-driven decision-making.<n>We propose a sequential forward-backward diffusion framework for adapted time series generation.<n>We derive rigorous statistical guarantees under a generic framework, then establish score approximation, score estimation, and distribution estimation results.
  arXiv  Detail & Related papers  (2026-06-04T10:59:24Z)
• SynCast: Synergizing Contradictions in Precipitation Nowcasting via Diffusion Sequential Preference Optimization [62.958457694151384]
  We introduce preference optimization into precipitation nowcasting for the first time, motivated by the success of reinforcement learning from human feedback in large language models.<n>In the first stage, the framework focuses on reducing FAR, training the model to effectively suppress false alarms.
  arXiv  Detail & Related papers  (2025-10-22T16:11:22Z)
• Preference Trajectory Modeling via Flow Matching for Sequential Recommendation [50.077447974294586]
  Sequential recommendation predicts each user's next item based on their historical interaction sequence.<n>FlowRec is a simple yet effective sequential recommendation framework.<n>We construct a personalized behavior-based prior distribution to replace Gaussian noise and learn a vector field to model user preference trajectories.
  arXiv  Detail & Related papers  (2025-08-25T02:55:42Z)
• Unified Flow Matching for Long Horizon Event Forecasting [3.0639815065447036]
  We propose a unified flow matching framework for marked temporal point processes.<n>By learning continuous-time flows for both components, our method generates coherent long horizon event trajectories without sequential decoding.<n>We evaluate our model on six real-world benchmarks and demonstrate significant improvements over autoregressive and diffusion-based baselines in both accuracy and generation efficiency.
  arXiv  Detail & Related papers  (2025-08-06T19:42:49Z)
• Self-attention-based Diffusion Model for Time-series Imputation in Partial Blackout Scenarios [23.160007389272575]
  Missing values in time series data can harm machine learning performance and introduce bias.<n>Previous work has tackled the imputation of missing data in random, complete blackouts and forecasting scenarios.<n>We introduce a two-stage imputation process using self-attention and diffusion processes to model feature and temporal correlations.
  arXiv  Detail & Related papers  (2025-03-03T16:58:15Z)
• Generative Regression Based Watch Time Prediction for Short-Video Recommendation [36.95095097454143]
  Watch time prediction has emerged as a pivotal task in short video recommendation systems.<n>Recent studies have attempted to address these issues by converting the continuous watch time estimation into an ordinal regression task.<n>We propose a novel Generative Regression (GR) framework that reformulates WTP as a sequence generation task.
  arXiv  Detail & Related papers  (2024-12-28T16:48:55Z)
• Long-Sequence Recommendation Models Need Decoupled Embeddings [49.410906935283585]
  We identify and characterize a neglected deficiency in existing long-sequence recommendation models.<n>A single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes.<n>We propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are learned separately to fully decouple attention and representation.
  arXiv  Detail & Related papers  (2024-10-03T15:45:15Z)
• On Your Mark, Get Set, Predict! Modeling Continuous-Time Dynamics of Cascades for Information Popularity Prediction [5.464598715181046]
  Key to accurately predicting information popularity lies in subtly modeling the underlying temporal information diffusion process. We propose ConCat, modeling the Continuous-time dynamics of Cascades for information popularity prediction. We conduct extensive experiments to evaluate ConCat on three real-world datasets.
  arXiv  Detail & Related papers  (2024-09-25T05:08:44Z)
• New User Event Prediction Through the Lens of Causal Inference [20.676353189313737]
  We propose a novel discrete event prediction framework for new users with limited history.<n>We treat the user event history as the "treatment" for future events and the user category as the key confounder.<n>We demonstrate the improved performance of the proposed framework with a numerical simulation study and two real-world applications.
  arXiv  Detail & Related papers  (2024-07-08T05:35:54Z)
• RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
  Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. We propose a concise and effective approach for CL with pre-trained models.
  arXiv  Detail & Related papers  (2023-07-05T12:49:02Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.