Higher-Order DeepTrails: Unified Approach to *Trails
- URL: http://arxiv.org/abs/2310.04477v2
- Date: Mon, 8 Jan 2024 11:40:32 GMT
- Title: Higher-Order DeepTrails: Unified Approach to *Trails
- Authors: Tobias Koopmann, Jan Pfister, Andr\'e Markus, Astrid Carolus, Carolin
Wienrich and Andreas Hotho
- Abstract summary: Analyzing, understanding, and describing human behavior is advantageous in different settings, such as web browsing or traffic navigation.
We propose to analyze entire sequences using autoregressive language models, as they are traditionally used to model higher-order dependencies in sequences.
We show that our approach can be easily adapted to model different settings introduced in previous work, namely HypTrails, MixedTrails and even SubTrails.
- Score: 7.270112855088838
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Analyzing, understanding, and describing human behavior is advantageous in
different settings, such as web browsing or traffic navigation. Understanding
human behavior naturally helps to improve and optimize the underlying
infrastructure or user interfaces. Typically, human navigation is represented
by sequences of transitions between states. Previous work suggests to use
hypotheses, representing different intuitions about the navigation to analyze
these transitions. To mathematically grasp this setting, first-order Markov
chains are used to capture the behavior, consequently allowing to apply
different kinds of graph comparisons, but comes with the inherent drawback of
losing information about higher-order dependencies within the sequences. To
this end, we propose to analyze entire sequences using autoregressive language
models, as they are traditionally used to model higher-order dependencies in
sequences. We show that our approach can be easily adapted to model different
settings introduced in previous work, namely HypTrails, MixedTrails and even
SubTrails, while at the same time bringing unique advantages: 1. Modeling
higher-order dependencies between state transitions, while 2. being able to
identify short comings in proposed hypotheses, and 3. naturally introducing a
unified approach to model all settings. To show the expressiveness of our
approach, we evaluate our approach on different synthetic datasets and conclude
with an exemplary analysis of a real-world dataset, examining the behavior of
users who interact with voice assistants.
Related papers
- AdaptSSR: Pre-training User Model with Augmentation-Adaptive
Self-Supervised Ranking [19.1857792382924]
We propose Augmentation-Supervised Ranking (AdaptSSR) to replace the contrastive learning task.
We adopt a multiple pairwise ranking loss which trains the user model to capture the similarity orders between the implicitly augmented view, the explicitly augmented view, and views from other users.
Experiments on both public and industrial datasets with six downstream tasks verify the effectiveness of AdaptSSR.
arXiv Detail & Related papers (2023-10-15T02:19:28Z) - Generalized Relation Modeling for Transformer Tracking [13.837171342738355]
One-stream trackers let the template interact with all parts inside the search region throughout all the encoder layers.
This could potentially lead to target-background confusion when the extracted feature representations are not sufficiently discriminative.
We propose a generalized relation modeling method based on adaptive token division.
Our method is superior to the two-stream and one-stream pipelines and achieves state-of-the-art performance on six challenging benchmarks with a real-time running speed.
arXiv Detail & Related papers (2023-03-29T10:29:25Z) - The Trade-off between Universality and Label Efficiency of
Representations from Contrastive Learning [32.15608637930748]
We show that there exists a trade-off between the two desiderata so that one may not be able to achieve both simultaneously.
We provide analysis using a theoretical data model and show that, while more diverse pre-training data result in more diverse features for different tasks, it puts less emphasis on task-specific features.
arXiv Detail & Related papers (2023-02-28T22:14:33Z) - Mutual Exclusivity Training and Primitive Augmentation to Induce
Compositionality [84.94877848357896]
Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models.
We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples.
We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
arXiv Detail & Related papers (2022-11-28T17:36:41Z) - Parameter Decoupling Strategy for Semi-supervised 3D Left Atrium
Segmentation [0.0]
We present a novel semi-supervised segmentation model based on parameter decoupling strategy to encourage consistent predictions from diverse views.
Our method has achieved a competitive result over the state-of-the-art semisupervised methods on the Atrial Challenge dataset.
arXiv Detail & Related papers (2021-09-20T14:51:42Z) - Contrastive Self-supervised Sequential Recommendation with Robust
Augmentation [101.25762166231904]
Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data.
Old and new issues remain, including data-sparsity and noisy data.
We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
arXiv Detail & Related papers (2021-08-14T07:15:25Z) - Sequence Adaptation via Reinforcement Learning in Recommender Systems [8.909115457491522]
We propose the SAR model, which learns the sequential patterns and adjusts the sequence length of user-item interactions in a personalized manner.
In addition, we optimize a joint loss function to align the accuracy of the sequential recommendations with the expected cumulative rewards of the critic network.
Our experimental evaluation on four real-world datasets demonstrates the superiority of our proposed model over several baseline approaches.
arXiv Detail & Related papers (2021-07-31T13:56:46Z) - TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning [87.38675639186405]
We propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion.
To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs.
arXiv Detail & Related papers (2021-05-17T15:33:25Z) - Learning Transferrable Parameters for Long-tailed Sequential User
Behavior Modeling [70.64257515361972]
We argue that focusing on tail users could bring more benefits and address the long tails issue.
Specifically, we propose a gradient alignment and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail.
arXiv Detail & Related papers (2020-10-22T03:12:02Z) - Document Ranking with a Pretrained Sequence-to-Sequence Model [56.44269917346376]
We show how a sequence-to-sequence model can be trained to generate relevance labels as "target words"
Our approach significantly outperforms an encoder-only model in a data-poor regime.
arXiv Detail & Related papers (2020-03-14T22:29:50Z) - Forecasting Sequential Data using Consistent Koopman Autoencoders [52.209416711500005]
A new class of physics-based methods related to Koopman theory has been introduced, offering an alternative for processing nonlinear dynamical systems.
We propose a novel Consistent Koopman Autoencoder model which, unlike the majority of existing work, leverages the forward and backward dynamics.
Key to our approach is a new analysis which explores the interplay between consistent dynamics and their associated Koopman operators.
arXiv Detail & Related papers (2020-03-04T18:24:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.