Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences

• URL: http://arxiv.org/abs/2509.12188v1
• Date: Mon, 15 Sep 2025 17:51:02 GMT
• Title: Event2Vec: A Geometric Approach to Learning Composable Representations of Event Sequences
• Authors: Antonin Sulc,
• Abstract summary: We introduce Event2Vec, a novel framework for learning representations of discrete event sequences.<n>We provide a theoretical analysis demonstrating that our model's learned representations in a Euclidean space converge to an ideal additive structure.<n>To address the limitations of Euclidean geometry for hierarchical data, we also introduce a variant of our model in hyperbolic space.
• Score: 0.15229257192293197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The study of neural representations, both in biological and artificial systems, is increasingly revealing the importance of geometric and topological structures. Inspired by this, we introduce Event2Vec, a novel framework for learning representations of discrete event sequences. Our model leverages a simple, additive recurrent structure to learn composable, interpretable embeddings. We provide a theoretical analysis demonstrating that, under specific training objectives, our model's learned representations in a Euclidean space converge to an ideal additive structure. This ensures that the representation of a sequence is the vector sum of its constituent events, a property we term the linear additive hypothesis. To address the limitations of Euclidean geometry for hierarchical data, we also introduce a variant of our model in hyperbolic space, which is naturally suited to embedding tree-like structures with low distortion. We present experiments to validate our hypothesis and demonstrate the benefits of each geometry, highlighting the improved performance of the hyperbolic model on hierarchical event sequences.

Related papers

• An Algebraic Representation Theorem for Linear GENEOs in Geometric Machine Learning [1.3425748364842416]
  Group Equivariant Non-Expansive Operators (GENEOs) have emerged as a powerful class of operators for encoding symmetries.<n>We introduce a new representation theorem for linear GENEOs acting between different perception pairs.<n>We also prove the compactness and convexity of the space of linear GENEOs.
  arXiv  Detail & Related papers  (2026-01-07T13:21:44Z)
• Learning Latent Graph Geometry via Fixed-Point Schrödinger-Type Activation: A Theoretical Study [1.1745324895296467]
  We develop a unified theoretical framework for neural architectures with internal representations evolving as stationary states of dissipative Schr"odinger-type dynamics on learned latent graphs.<n>We prove existence, uniqueness, and smooth dependence of equilibria, and show that the dynamics are equivalent under the Bloch map to norm-preserving Landau--Lifshitz flows.<n>The resulting model class provides a compact, geometrically interpretable, and analytically tractable foundation for learning latent graph geometry via fixed-point Schr"odinger-type activations.
  arXiv  Detail & Related papers  (2025-07-27T00:35:15Z)
• TokenBlowUp: Resolving Representational Singularities in LLM Token Spaces via Monoidal Transformations [1.3824176915623292]
  Recent work has provided compelling evidence challenging the foundational manifold hypothesis for the token embedding spaces of Large Language Models.<n>We formalize this problem in the language of scheme theory and propose a rigorous resolution by applying the scheme-theoretic blow-up at each singular point.<n>We prove a formal theorem guaranteeing the geometric regularization of this new space, showing that the original pathologies are resolved.
  arXiv  Detail & Related papers  (2025-07-26T02:39:54Z)
• Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning [73.18052192964349]
  We develop a theoretical framework that explains how discrete symbolic structures can emerge naturally from continuous neural network training dynamics.<n>By lifting neural parameters to a measure space and modeling training as Wasserstein gradient flow, we show that under geometric constraints, the parameter measure $mu_t$ undergoes two concurrent phenomena.
  arXiv  Detail & Related papers  (2025-06-26T22:40:30Z)
• SVarM: Linear Support Varifold Machines for Classification and Regression on Geometric Data [4.212663349859165]
  This work proposes textitSVarM to exploit varifold representations of shapes as measures and their duality with test functions $h:mathbbRn times Sn-1 rightarrow mathbbR$.<n>We develop classification and regression models on shape datasets by introducing a neural network-based representation of the trainable test function $h$.
  arXiv  Detail & Related papers  (2025-06-01T21:55:15Z)
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• On the Origins of Linear Representations in Large Language Models [51.88404605700344]
  We introduce a simple latent variable model to formalize the concept dynamics of the next token prediction. Experiments show that linear representations emerge when learning from data matching the latent variable model. We additionally confirm some predictions of the theory using the LLaMA-2 large language model.
  arXiv  Detail & Related papers  (2024-03-06T17:17:36Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Semantically-informed Hierarchical Event Modeling [14.00844847268286]
  We present a novel, doubly hierarchical, semi-supervised event modeling framework. Our approach consists of multiple layers of structured latent variables, where each successive layer compresses and abstracts the previous layers. We demonstrate that our approach is able to out-perform the previous state-of-the-art approaches by up to 8.5%.
  arXiv  Detail & Related papers  (2022-12-20T18:51:23Z)
• A Fully Hyperbolic Neural Model for Hierarchical Multi-Class Classification [7.8176853587105075]
  Hyperbolic spaces offer a mathematically appealing approach for learning hierarchical representations of symbolic data. This work proposes a fully hyperbolic model for multi-class multi-label classification, which performs all operations in hyperbolic space. A thorough analysis sheds light on the impact of each component in the final prediction and showcases its ease of integration with Euclidean layers.
  arXiv  Detail & Related papers  (2020-10-05T14:42:56Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.