Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention

• URL: http://arxiv.org/abs/2303.00280v3
• Date: Mon, 28 Oct 2024 14:13:29 GMT
• Title: Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention
• Authors: Elizaveta Kovtun, Galina Boeva, Andrey Shulga, Alexey Zaytsev,
• Abstract summary: Anticipation of the label set for the future event holds significant value. The proposed model is called Label-Attention NETwork, or LANET.
• Score: 2.487894881721314
• License:
• Abstract: Most user-related data can be represented as a sequence of events associated with a timestamp and a collection of categorical labels. For example, the purchased basket of goods and the time of buying fully characterize the event of the store visit. Anticipation of the label set for the future event called the problem of temporal sets prediction, holds significant value, especially in such high-stakes industries as finance and e-commerce. A fundamental challenge of this task is the joint consideration of the temporal nature of events and label relations within sets. The existing models fail to capture complex time and label dependencies due to ineffective representation of historical information initially. We aim to address this shortcoming by presenting the framework with a specific way to aggregate the observed information into time- and set structure-aware views prior to transferring it into main architecture blocks. Our strong emphasis on input arrangement facilitates the subsequent efficient learning of label interactions. The proposed model is called Label-Attention NETwork, or LANET. We conducted experiments on four different datasets and made a comparison with four established models, including SOTA, in this area. The experimental results suggest that LANET provides significantly better quality than any other model, achieving an improvement up to $65 \%$ in terms of weighted F1 metric compared to the closest competitor. Moreover, we contemplate causal relationships between labels in our work, as well as a thorough study of LANET components' influence on performance. We provide an implementation of LANET to encourage its wider usage.

Related papers

• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Exploring the Limits of Historical Information for Temporal Knowledge Graph Extrapolation [59.417443739208146]
  We propose a new event forecasting model based on a novel training framework of historical contrastive learning. CENET learns both the historical and non-historical dependency to distinguish the most potential entities. We evaluate our proposed model on five benchmark graphs.
  arXiv  Detail & Related papers  (2023-08-29T03:26:38Z)
• Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMs [16.700089674927348]
  Large Language Models (LLMs) have shown surprising results in numerous natural language processing tasks. This paper investigates their powerful learning capabilities in natural language and effectiveness in predicting relations between product types with limited labeled data. Our results show that LLMs significantly outperform existing KG completion models in relation labeling for e-commerce KGs and exhibit performance strong enough to replace human labeling.
  arXiv  Detail & Related papers  (2023-05-17T00:08:36Z)
• Enhancing Label Sharing Efficiency in Complementary-Label Learning with Label Augmentation [92.4959898591397]
  We analyze the implicit sharing of complementary labels on nearby instances during training. We propose a novel technique that enhances the sharing efficiency via complementary-label augmentation. Our results confirm that complementary-label augmentation can systematically improve empirical performance over state-of-the-art CLL models.
  arXiv  Detail & Related papers  (2023-05-15T04:43:14Z)
• Unified Visual Relationship Detection with Vision and Language Models [89.77838890788638]
  This work focuses on training a single visual relationship detector predicting over the union of label spaces from multiple datasets. We propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models. Empirical results on both human-object interaction detection and scene-graph generation demonstrate the competitive performance of our model.
  arXiv  Detail & Related papers  (2023-03-16T00:06:28Z)
• Temporal Knowledge Graph Reasoning with Historical Contrastive Learning [24.492458924487863]
  We propose a new event forecasting model called Contrastive Event Network (CENET) CENET learns both the historical and non-historical dependency to distinguish the most potential entities that can best match the given query. During the inference process, CENET employs a mask-based strategy to generate the final results.
  arXiv  Detail & Related papers  (2022-11-20T08:32:59Z)
• Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis [39.770652220521384]
  State-of-the-art model for structured sentiment analysis casts the task as a dependency parsing problem. Label proportions for span prediction and span relation prediction are imbalanced. Two nodes in a dependency graph cannot have multiple arcs, therefore some overlapped sentiments cannot be recognized.
  arXiv  Detail & Related papers  (2022-03-21T08:23:03Z)
• Enabling Efficiency-Precision Trade-offs for Label Trees in Extreme Classification [43.840626501982314]
  Extreme multi-label classification (XMC) aims to learn a model that can tag data points with a subset of relevant labels from an extremely large label set. We propose an efficient information theory inspired algorithm to construct intermediary operating points that trade off between the benefits of both. Our method can reduce a proxy for expected latency by up to 28% while maintaining the same accuracy as Parabel.
  arXiv  Detail & Related papers  (2021-06-01T19:02:09Z)
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection [101.56529337489417]
  We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of human, action, object> in images. We argue that multi-level consistencies among objects, actions and interactions are strong cues for generating semantic representations of rare or previously unseen HOIs. Our model takes visual features of candidate human-object pairs and word embeddings of HOI labels as inputs, maps them into visual-semantic joint embedding space and obtains detection results by measuring their similarities.
  arXiv  Detail & Related papers  (2020-08-14T09:11:18Z)
• Social Adaptive Module for Weakly-supervised Group Activity Recognition [143.68241396839062]
  This paper presents a new task named weakly-supervised group activity recognition (GAR) It differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data. This eases us to collect and annotate a large-scale NBA dataset and thus raise new challenges to GAR.
  arXiv  Detail & Related papers  (2020-07-18T16:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.