Looking around you: external information enhances representations for event sequences

• URL: http://arxiv.org/abs/2502.10205v2
• Date: Mon, 16 Jun 2025 13:14:34 GMT
• Title: Looking around you: external information enhances representations for event sequences
• Authors: Maria Kovaleva, Petr Sokerin, Pavel Tikhomirov, Alexey Zaytsev,
• Abstract summary: Representation learning produces models in different domains, such as store purchases, client transactions, and general people's behaviour.<n>We develop a method that aggregates information from multiple user representations, augmenting a specific user for a scenario of multiple co-occurring event sequences.<n>Our study considers diverse aggregation approaches, ranging from simple pooling techniques to trainable attention-based Kernel attention aggregation.
• Score: 2.1879059908547482
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Representation learning produces models in different domains, such as store purchases, client transactions, and general people's behaviour. However, such models for event sequences usually process each sequence in isolation, ignoring context from ones that co-occur in time. This limitation is particularly problematic in domains with fast-evolving conditions, like finance and e-commerce, or when certain sequences lack recent events. We develop a method that aggregates information from multiple user representations, augmenting a specific user for a scenario of multiple co-occurring event sequences, achieving better quality than processing each sequence independently. Our study considers diverse aggregation approaches, ranging from simple pooling techniques to trainable attention-based Kernel attention aggregation, that can highlight more complex information flow from other users. The proposed methods operate on top of an existing encoder and support its efficient fine-tuning. Across six diverse event sequence datasets (finance, e-commerce, education, etc.) and downstream tasks, Kernel attention improves ROC-AUC scores, both with and without fine-tuning, while mean pooling yields a smaller but still significant gain.

Related papers

• STAR-Rec: Making Peace with Length Variance and Pattern Diversity in Sequential Recommendation [61.320991769685065]
  STAR-Rec is a novel architecture that combines preference-aware attention and state-space modeling.<n>We show that STAR-Rec consistently outperforms state-of-the-art sequential recommendation methods.
  arXiv  Detail & Related papers  (2025-05-06T12:40:38Z)
• Recommendation System in Advertising and Streaming Media: Unsupervised Data Enhancement Sequence Suggestions [2.9633211091806997]
  We introduce a novel framework, Global Unsupervised Data-Augmentation (UDA4SR), which adopts a graph contrastive learning perspective to generate robust item embeddings for sequential recommendation. Our approach begins by integrating Generative Adrial Networks (GANs) for data augmentation, which serves as the first step to enhance the diversity and richness of the training data. To model users' dynamic and diverse interests more effectively, we enhance the CapsNet module with a novel target-attention mechanism.
  arXiv  Detail & Related papers  (2025-03-23T06:30:48Z)
• Multimodal Difference Learning for Sequential Recommendation [5.243083216855681]
  We argue that user interests and item relationships vary across different modalities.<n>We propose a novel Multimodal Learning framework for Sequential Recommendation, MDSRec.<n>Results on five real-world datasets demonstrate the superiority of MDSRec over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-12-11T05:08:19Z)
• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms.<n>Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction.<n>We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• Long-Sequence Recommendation Models Need Decoupled Embeddings [49.410906935283585]
  We identify and characterize a neglected deficiency in existing long-sequence recommendation models. A single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes. We propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are learned separately to fully decouple attention and representation.
  arXiv  Detail & Related papers  (2024-10-03T15:45:15Z)
• Uniting contrastive and generative learning for event sequences models [51.547576949425604]
  This study investigates the integration of two self-supervised learning techniques - instance-wise contrastive learning and a generative approach based on restoring masked events in latent space.<n> Experiments conducted on several public datasets, focusing on sequence classification and next-event type prediction, show that the integrated method achieves superior performance compared to individual approaches.
  arXiv  Detail & Related papers  (2024-08-19T13:47:17Z)
• Sample Enrichment via Temporary Operations on Subsequences for Sequential Recommendation [15.718287580146272]
  We propose a novel model-agnostic and highly generic framework for sequential recommendation called sample enrichment via temporary operations on subsequences (SETO) We highlight our SETO's effectiveness and versatility over multiple representative and state-of-the-art sequential recommendation models across multiple real-world datasets.
  arXiv  Detail & Related papers  (2024-07-25T06:22:08Z)
• SEMINAR: Search Enhanced Multi-modal Interest Network and Approximate Retrieval for Lifelong Sequential Recommendation [16.370075234443245]
  We propose a unified lifelong multi-modal sequence model called SEMINAR-Search Enhanced Multi-Modal Interest Network and Approximate Retrieval. Specifically, a network called Pretraining Search Unit learns the lifelong sequences of multi-modal query-item pairs in a pretraining-finetuning manner. To accelerate the online retrieval speed of multi-modal embedding, we propose a multi-modal codebook-based product quantization strategy.
  arXiv  Detail & Related papers  (2024-07-15T13:33:30Z)
• Enhancing Few-shot NER with Prompt Ordering based Data Augmentation [59.69108119752584]
  We propose a Prompt Ordering based Data Augmentation (PODA) method to improve the training of unified autoregressive generation frameworks. Experimental results on three public NER datasets and further analyses demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-05-19T16:25:43Z)
• Robust Detection of Lead-Lag Relationships in Lagged Multi-Factor Models [61.10851158749843]
  Key insights can be obtained by discovering lead-lag relationships inherent in the data. We develop a clustering-driven methodology for robust detection of lead-lag relationships in lagged multi-factor models.
  arXiv  Detail & Related papers  (2023-05-11T10:30:35Z)
• Continuous-time convolutions model of event sequences [46.3471121117337]
  Event sequences are non-uniform and sparse, making traditional models unsuitable. We propose COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time. COTIC outperforms existing models in predicting the next event time and type, achieving an average rank of 1.5 compared to 3.714 for the nearest competitor.
  arXiv  Detail & Related papers  (2023-02-13T10:34:51Z)
• Towards Lightweight Cross-domain Sequential Recommendation via External Attention-enhanced Graph Convolution Network [7.1102362215550725]
  Cross-domain Sequential Recommendation (CSR) depicts the evolution of behavior patterns for overlapped users by modeling their interactions from multiple domains. We introduce a lightweight external attention-enhanced GCN-based framework to solve the above challenges, namely LEA-GCN. To further alleviate the framework structure and aggregate the user-specific sequential pattern, we devise a novel dual-channel External Attention (EA) component.
  arXiv  Detail & Related papers  (2023-02-07T03:06:29Z)
• Time Interval-enhanced Graph Neural Network for Shared-account Cross-domain Sequential Recommendation [44.34610028544989]
  Shared-account Cross-domain Sequential Recommendation (SCSR) task aims to recommend the next item via leveraging the mixed user behaviors in multiple domains. Existing works on SCSR mainly rely on mining sequential patterns via Recurrent Neural Network (RNN)-based models. We propose a new graph-based solution, namely TiDA-GCN, to address the above challenges.
  arXiv  Detail & Related papers  (2022-06-16T10:06:01Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Neural Hierarchical Factorization Machines for User's Event Sequence Analysis [21.13650689194003]
  We consider a two-level structure for capturing the hierarchical information over user's event sequence. Our model achieves significantly better performance compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2021-12-31T04:08:55Z)
• COHORTNEY: Deep Clustering for Heterogeneous Event Sequences [9.811178291117496]
  Clustering of event sequences is widely applicable in domains such as healthcare, marketing, and finance. We propose COHORTNEY as a novel deep learning method for clustering heterogeneous event sequences. Our results show that COHORTNEY vastly outperforms in speed and cluster quality the state-of-the-art algorithm for clustering event sequences.
  arXiv  Detail & Related papers  (2021-04-03T16:12:21Z)
• Sparse-Interest Network for Sequential Recommendation [78.83064567614656]
  We propose a novel textbfSparse textbfInterest textbfNEtwork (SINE) for sequential recommendation. Our sparse-interest module can adaptively infer a sparse set of concepts for each user from the large concept pool. SINE can achieve substantial improvement over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-18T11:03:48Z)
• Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection [63.825781848587376]
  We propose OC4Seq, a one-class recurrent neural network for detecting anomalies in discrete event sequences. Specifically, OC4Seq embeds the discrete event sequences into latent spaces, where anomalies can be easily detected.
  arXiv  Detail & Related papers  (2020-08-31T04:48:22Z)
• Adversarial Encoder-Multi-Task-Decoder for Multi-Stage Processes [5.933303832684138]
  In multi-stage processes, decisions occur in an ordered sequence of stages. We introduce a framework that combines adversarial autoencoders (AAE), multi-task learning (MTL), and multi-label semi-supervised learning (MLSSL) Using real-world data from different domains, we show that our approach outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-15T19:30:31Z)
• CoLES: Contrastive Learning for Event Sequences with Self-Supervision [63.3568071938238]
  We address the problem of self-supervised learning on discrete event sequences generated by real-world users. We propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains.
  arXiv  Detail & Related papers  (2020-02-19T15:15:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.