Looking around you: external information enhances representations for event sequences

• URL: http://arxiv.org/abs/2502.10205v1
• Date: Fri, 14 Feb 2025 14:59:37 GMT
• Title: Looking around you: external information enhances representations for event sequences
• Authors: Maria Kovaleva, Petr Sokerin, Sofia Krehova, Alexey Zaytsev,
• Abstract summary: We propose a method that aggregates information from multiple user representations augmenting a specific user one. Our study considers diverse aggregation approaches, ranging from simple pooling techniques to trainable attention-based approaches. The proposed method operates atop an existing encoder and supports its efficient fine-tuning.
• Score: 2.1879059908547482
• License:
• Abstract: Representation learning produces models in different domains, such as store purchases, client transactions, and general people's behaviour. However, such models for sequential data usually process a single sequence, ignoring context from other relevant ones, even in domains with rapidly changing external environments like finance or misguiding the prediction for a user with no recent events. We are the first to propose a method that aggregates information from multiple user representations augmenting a specific user one for a scenario of multiple co-occurring event sequences. Our study considers diverse aggregation approaches, ranging from simple pooling techniques to trainable attention-based approaches, especially Kernel attention aggregation, that can highlight more complex information flow from other users. The proposed method operates atop an existing encoder and supports its efficient fine-tuning. Across considered datasets of financial transactions and downstream tasks, Kernel attention improves ROC AUC scores, both with and without fine-tuning, while mean pooling yields a smaller but still significant gain.

Related papers

• Multimodal Difference Learning for Sequential Recommendation [5.243083216855681]
  We argue that user interests and item relationships vary across different modalities. We propose a novel Multimodal Learning framework for Sequential Recommendation, MDSRec. Results on five real-world datasets demonstrate the superiority of MDSRec over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-12-11T05:08:19Z)
• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms. Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction. We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• Uniting contrastive and generative learning for event sequences models [51.547576949425604]
  This study investigates the integration of two self-supervised learning techniques - instance-wise contrastive learning and a generative approach based on restoring masked events in latent space. Experiments conducted on several public datasets, focusing on sequence classification and next-event type prediction, show that the integrated method achieves superior performance compared to individual approaches.
  arXiv  Detail & Related papers  (2024-08-19T13:47:17Z)
• Sample Enrichment via Temporary Operations on Subsequences for Sequential Recommendation [15.718287580146272]
  We propose a novel model-agnostic and highly generic framework for sequential recommendation called sample enrichment via temporary operations on subsequences (SETO) We highlight our SETO's effectiveness and versatility over multiple representative and state-of-the-art sequential recommendation models across multiple real-world datasets.
  arXiv  Detail & Related papers  (2024-07-25T06:22:08Z)
• Large-scale Fully-Unsupervised Re-Identification [78.47108158030213]
  We propose two strategies to learn from large-scale unlabeled data. The first strategy performs a local neighborhood sampling to reduce the dataset size in each without violating neighborhood relationships. A second strategy leverages a novel Re-Ranking technique, which has a lower time upper bound complexity and reduces the memory complexity from O(n2) to O(kn) with k n.
  arXiv  Detail & Related papers  (2023-07-26T16:19:19Z)
• Enhancing Few-shot NER with Prompt Ordering based Data Augmentation [59.69108119752584]
  We propose a Prompt Ordering based Data Augmentation (PODA) method to improve the training of unified autoregressive generation frameworks. Experimental results on three public NER datasets and further analyses demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-05-19T16:25:43Z)
• Towards Lightweight Cross-domain Sequential Recommendation via External Attention-enhanced Graph Convolution Network [7.1102362215550725]
  Cross-domain Sequential Recommendation (CSR) depicts the evolution of behavior patterns for overlapped users by modeling their interactions from multiple domains. We introduce a lightweight external attention-enhanced GCN-based framework to solve the above challenges, namely LEA-GCN. To further alleviate the framework structure and aggregate the user-specific sequential pattern, we devise a novel dual-channel External Attention (EA) component.
  arXiv  Detail & Related papers  (2023-02-07T03:06:29Z)
• Time Interval-enhanced Graph Neural Network for Shared-account Cross-domain Sequential Recommendation [44.34610028544989]
  Shared-account Cross-domain Sequential Recommendation (SCSR) task aims to recommend the next item via leveraging the mixed user behaviors in multiple domains. Existing works on SCSR mainly rely on mining sequential patterns via Recurrent Neural Network (RNN)-based models. We propose a new graph-based solution, namely TiDA-GCN, to address the above challenges.
  arXiv  Detail & Related papers  (2022-06-16T10:06:01Z)
• Neural Hierarchical Factorization Machines for User's Event Sequence Analysis [21.13650689194003]
  We consider a two-level structure for capturing the hierarchical information over user's event sequence. Our model achieves significantly better performance compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2021-12-31T04:08:55Z)
• Sparse-Interest Network for Sequential Recommendation [78.83064567614656]
  We propose a novel textbfSparse textbfInterest textbfNEtwork (SINE) for sequential recommendation. Our sparse-interest module can adaptively infer a sparse set of concepts for each user from the large concept pool. SINE can achieve substantial improvement over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-02-18T11:03:48Z)
• CoLES: Contrastive Learning for Event Sequences with Self-Supervision [63.3568071938238]
  We address the problem of self-supervised learning on discrete event sequences generated by real-world users. We propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains.
  arXiv  Detail & Related papers  (2020-02-19T15:15:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.