Uniting contrastive and generative learning for event sequences models

• URL: http://arxiv.org/abs/2408.09995v2
• Date: Mon, 23 Sep 2024 11:46:00 GMT
• Title: Uniting contrastive and generative learning for event sequences models
• Authors: Aleksandr Yugay, Alexey Zaytsev,
• Abstract summary: This study investigates the integration of two self-supervised learning techniques - instance-wise contrastive learning and a generative approach based on restoring masked events in latent space. Experiments conducted on several public datasets, focusing on sequence classification and next-event type prediction, show that the integrated method achieves superior performance compared to individual approaches.
• Score: 51.547576949425604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-quality representation of transactional sequences is vital for modern banking applications, including risk management, churn prediction, and personalized customer offers. Different tasks require distinct representation properties: local tasks benefit from capturing the client's current state, while global tasks rely on general behavioral patterns. Previous research has demonstrated that various self-supervised approaches yield representations that better capture either global or local qualities. This study investigates the integration of two self-supervised learning techniques - instance-wise contrastive learning and a generative approach based on restoring masked events in latent space. The combined approach creates representations that balance local and global transactional data characteristics. Experiments conducted on several public datasets, focusing on sequence classification and next-event type prediction, show that the integrated method achieves superior performance compared to individual approaches and demonstrates synergistic effects. These findings suggest that the proposed approach offers a robust framework for advancing event sequences representation learning in the financial sector.

Related papers

• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection [33.210575826086654]
  We present a weakly-supervised RGB-D salient object detection model via supervision. We focus on effective multimodal representation learning via inter-modal mutual information regularization.
  arXiv  Detail & Related papers  (2023-06-06T12:36:57Z)
• An Operational Perspective to Fairness Interventions: Where and How to Intervene [9.833760837977222]
  We present a holistic framework for evaluating and contextualizing fairness interventions. We demonstrate our framework with a case study on predictive parity. We find predictive parity is difficult to achieve without using group data.
  arXiv  Detail & Related papers  (2023-02-03T07:04:33Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Multivariate Business Process Representation Learning utilizing Gramian Angular Fields and Convolutional Neural Networks [0.0]
  Learning meaningful representations of data is an important aspect of machine learning. For predictive process analytics, it is essential to have all explanatory characteristics of a process instance available. We propose a novel approach for representation learning of business process instances.
  arXiv  Detail & Related papers  (2021-06-15T10:21:14Z)
• Information Maximization Clustering via Multi-View Self-Labelling [9.947717243638289]
  We propose a novel single-phase clustering method that simultaneously learns meaningful representations and assigns the corresponding annotations. This is achieved by integrating a discrete representation into the self-supervised paradigm through a net. Our empirical results show that the proposed framework outperforms state-of-the-art techniques with the average accuracy of 89.1% and 49.0%, respectively.
  arXiv  Detail & Related papers  (2021-03-12T16:04:41Z)
• Stochastic Action Prediction for Imitation Learning [1.6385815610837169]
  Imitation learning is a data-driven approach to acquiring skills that relies on expert demonstrations to learn a policy that maps observations to actions. We demonstrate inherentity in demonstrations collected for tasks including line following with a remote-controlled car. We find that accounting for adversariality in the expert data leads to substantial improvement in the success rate of task completion.
  arXiv  Detail & Related papers  (2020-12-26T08:02:33Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)
• Adaptive Object Detection with Dual Multi-Label Prediction [78.69064917947624]
  We propose a novel end-to-end unsupervised deep domain adaptation model for adaptive object detection. The model exploits multi-label prediction to reveal the object category information in each image. We introduce a prediction consistency regularization mechanism to assist object detection.
  arXiv  Detail & Related papers  (2020-03-29T04:23:22Z)
• CoLES: Contrastive Learning for Event Sequences with Self-Supervision [63.3568071938238]
  We address the problem of self-supervised learning on discrete event sequences generated by real-world users. We propose a new method "CoLES", which adapts contrastive learning, previously used for audio and computer vision domains.
  arXiv  Detail & Related papers  (2020-02-19T15:15:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.