Representation Learning of Tangled Key-Value Sequence Data for Early Classification

• URL: http://arxiv.org/abs/2404.07454v1
• Date: Thu, 11 Apr 2024 03:23:15 GMT
• Title: Representation Learning of Tangled Key-Value Sequence Data for Early Classification
• Authors: Tao Duan, Junzhou Zhao, Shuo Zhang, Jing Tao, Pinghui Wang,
• Abstract summary: Key-value sequence data has become ubiquitous and naturally appears in a variety of real-world applications. Classifying these key-value sequences is important in many scenarios such as user profiling and malicious applications identification. In many time-sensitive scenarios, besides the requirement of classifying a key-value sequence accurately, it is also desired to classify a key-value sequence early.
• Score: 19.943311002522154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Key-value sequence data has become ubiquitous and naturally appears in a variety of real-world applications, ranging from the user-product purchasing sequences in e-commerce, to network packet sequences forwarded by routers in networking. Classifying these key-value sequences is important in many scenarios such as user profiling and malicious applications identification. In many time-sensitive scenarios, besides the requirement of classifying a key-value sequence accurately, it is also desired to classify a key-value sequence early, in order to respond fast. However, these two goals are conflicting in nature, and it is challenging to achieve them simultaneously. In this work, we formulate a novel tangled key-value sequence early classification problem, where a tangled key-value sequence is a mixture of several concurrent key-value sequences with different keys. The goal is to classify each individual key-value sequence sharing a same key both accurately and early. To address this problem, we propose a novel method, i.e., Key-Value sequence Early Co-classification (KVEC), which leverages both inner- and inter-correlations of items in a tangled key-value sequence through key correlation and value correlation to learn a better sequence representation. Meanwhile, a time-aware halting policy decides when to stop the ongoing key-value sequence and classify it based on current sequence representation. Experiments on both real-world and synthetic datasets demonstrate that our method outperforms the state-of-the-art baselines significantly. KVEC improves the prediction accuracy by up to $4.7 - 17.5\%$ under the same prediction earliness condition, and improves the harmonic mean of accuracy and earliness by up to $3.7 - 14.0\%$.

Related papers

• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms. Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction. We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• Ego-Network Transformer for Subsequence Classification in Time Series Data [36.591480151951515]
  Real-world time series data often contain foreground subsequences intertwined with background subsequences. We propose a novel subsequence classification method that represents each subsequence as an ego-network. Our method outperforms the baseline on 104 out of 158 datasets.
  arXiv  Detail & Related papers  (2023-11-05T04:21:42Z)
• Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network [52.29330138835208]
  Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images. We propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide message passing.
  arXiv  Detail & Related papers  (2023-07-04T02:50:44Z)
• A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images [8.73248722579337]
  We present a new question-answering (QA) based key-value pair extraction approach, called KVPFormer. We propose a coarse-to-fine answer prediction approach to achieve higher answer prediction accuracy. Our proposed Ours achieves state-of-the-art results on FUNSD and XFUND datasets, outperforming the previous best-performing method by 7.2% and 13.2% in F1 score, respectively.
  arXiv  Detail & Related papers  (2023-04-17T02:55:31Z)
• Semi-Structured Object Sequence Encoders [9.257633944317735]
  We focus on the problem of developing a structure-aware input representation for semi-structured object sequences. This type of data is often represented as a sequence of sets of key-value pairs over time. We propose a two-part approach, which first considers each key independently and encodes a representation of its values over time.
  arXiv  Detail & Related papers  (2023-01-03T09:19:41Z)
• Discrete Key-Value Bottleneck [95.61236311369821]
  Deep neural networks perform well on classification tasks where data streams are i.i.d. and labeled data is abundant. One powerful approach that has addressed this challenge involves pre-training of large encoders on volumes of readily available data, followed by task-specific tuning. Given a new task, however, updating the weights of these encoders is challenging as a large number of weights needs to be fine-tuned, and as a result, they forget information about the previous tasks. We propose a model architecture to address this issue, building upon a discrete bottleneck containing pairs of separate and learnable key-value codes.
  arXiv  Detail & Related papers  (2022-07-22T17:52:30Z)
• Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences [24.963828650935913]
  We propose NEUROSEQRET which learns to retrieve and rank a relevant set of continuous-time event sequences for a given query sequence. We develop two variants of the relevance model which offer a tradeoff between accuracy and efficiency. Our experiments with several datasets show the significant accuracy boost of NEUROSEQRET beyond several baselines.
  arXiv  Detail & Related papers  (2022-02-17T11:16:31Z)
• Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model [93.9943278892735]
  Key problem in protein sequence representation learning is to capture the co-evolutionary information reflected by the inter-residue co-variation in the sequences. We propose a novel method to capture this information directly by pre-training via a dedicated language model, i.e., Pairwise Masked Language Model (PMLM) Our result shows that the proposed method can effectively capture the interresidue correlations and improves the performance of contact prediction by up to 9% compared to the baseline.
  arXiv  Detail & Related papers  (2021-10-29T04:01:32Z)
• MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction [48.55908127994688]
  We propose a novel key-value matching model based on a graph neural network for VIE (MatchVIE) Through key-value matching based on relevancy evaluation, the proposed MatchVIE can bypass the recognitions to various semantics. We introduce a simple but effective operation, Num2Vec, to tackle the instability of encoded values.
  arXiv  Detail & Related papers  (2021-06-24T12:06:29Z)
• One2Set: Generating Diverse Keyphrases as a Set [12.670421834049668]
  We propose a new training paradigm One2Set without predefining an order to the keyphrases. We propose a $K$-step target assignment mechanism via bipartite matching, which greatly increases the diversity and reduces the duplication ratio of generated keyphrases.
  arXiv  Detail & Related papers  (2021-05-24T07:29:47Z)
• Document Ranking with a Pretrained Sequence-to-Sequence Model [56.44269917346376]
  We show how a sequence-to-sequence model can be trained to generate relevance labels as "target words" Our approach significantly outperforms an encoder-only model in a data-poor regime.
  arXiv  Detail & Related papers  (2020-03-14T22:29:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.