Deep Learning on Attributed Sequences

• URL: http://arxiv.org/abs/2201.09199v1
• Date: Sun, 23 Jan 2022 06:54:31 GMT
• Title: Deep Learning on Attributed Sequences
• Authors: Zhongfang Zhuang
• Abstract summary: We focus on analyzing and building deep learning models for four new problems on attributed sequences. Our experiments on real-world datasets demonstrate that the proposed solutions significantly improve the performance of each task.
• Score: 0.38707695363745215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent research in feature learning has been extended to sequence data, where each instance consists of a sequence of heterogeneous items with a variable length. However, in many real-world applications, the data exists in the form of attributed sequences, which is composed of a set of fixed-size attributes and variable-length sequences with dependencies between them. In the attributed sequence context, feature learning remains challenging due to the dependencies between sequences and their associated attributes. In this dissertation, we focus on analyzing and building deep learning models for four new problems on attributed sequences. Our extensive experiments on real-world datasets demonstrate that the proposed solutions significantly improve the performance of each task over the state-of-the-art methods on attributed sequences.

Related papers

• Learning to Search Effective Example Sequences for In-Context Learning [6.532919612658209]
  We introduce Beam Search-based Example Sequence Constructor (BESC), a novel method for learning to construct optimal example sequences. BESC addresses all key factors involved in sequence selection by considering them jointly during inference, while incrementally building the sequence. Experiments across various datasets and language models show notable improvements in performance.
  arXiv  Detail & Related papers  (2025-03-11T04:24:59Z)
• Behavioral Sequence Modeling with Ensemble Learning [8.241486511994202]
  We present a framework for sequence modeling using Ensembles of Hidden Markov Models. Our ensemble-based scoring method enables robust comparison across sequences of different lengths. We demonstrate the effectiveness of our method with results on a longitudinal human behavior dataset.
  arXiv  Detail & Related papers  (2024-11-04T15:34:28Z)
• ComboStoc: Combinatorial Stochasticity for Diffusion Generative Models [65.82630283336051]
  We show that the space spanned by the combination of dimensions and attributes is insufficiently sampled by existing training scheme of diffusion generative models. We present a simple fix to this problem by constructing processes that fully exploit the structures, hence the name ComboStoc.
  arXiv  Detail & Related papers  (2024-05-22T15:23:10Z)
• Latent Processes Identification From Multi-View Time Series [17.33428123777779]
  We propose a novel framework that employs the contrastive learning technique to invert the data generative process for enhanced identifiability. MuLTI integrates a permutation mechanism that merges corresponding overlapped variables by the establishment of an optimal transport formula.
  arXiv  Detail & Related papers  (2023-05-14T14:21:58Z)
• Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality [84.94877848357896]
  Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models. We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples. We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
  arXiv  Detail & Related papers  (2022-11-28T17:36:41Z)
• Dynamic Latent Separation for Deep Learning [67.62190501599176]
  A core problem in machine learning is to learn expressive latent variables for model prediction on complex data. Here, we develop an approach that improves expressiveness, provides partial interpretation, and is not restricted to specific applications.
  arXiv  Detail & Related papers  (2022-10-07T17:56:53Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Self-Attention Neural Bag-of-Features [103.70855797025689]
  We build on the recently introduced 2D-Attention and reformulate the attention learning methodology. We propose a joint feature-temporal attention mechanism that learns a joint 2D attention mask highlighting relevant information.
  arXiv  Detail & Related papers  (2022-01-26T17:54:14Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Multi-Scale Feature and Metric Learning for Relation Extraction [15.045539169021092]
  We propose a multi-scale feature and metric learning framework for relation extraction. Specifically, we first develop a multi-scale convolutional neural network to aggregate the non-successive mainstays in the lexical sequence. We also design a multi-scale graph convolutional network which can increase the receptive field towards specific syntactic roles.
  arXiv  Detail & Related papers  (2021-07-28T15:14:36Z)
• Top-$k$ Regularization for Supervised Feature Selection [11.927046591097623]
  We introduce a novel, simple yet effective regularization approach, named top-$k$ regularization, to supervised feature selection. We show that the top-$k$ regularization is effective and stable for supervised feature selection.
  arXiv  Detail & Related papers  (2021-06-04T01:12:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.