In-Context Learning of Temporal Point Processes with Foundation Inference Models

• URL: http://arxiv.org/abs/2509.24762v1
• Date: Mon, 29 Sep 2025 13:28:06 GMT
• Title: In-Context Learning of Temporal Point Processes with Foundation Inference Models
• Authors: David Berghaus, Patrick Seifner, Kostadin Cvejoski, César Ojeda, Ramsés J. Sánchez,
• Abstract summary: Current approaches to MTPP inference rely on training separate, specialized models for each target system.<n>We pretrain a deep neural network to infer, in-context, the conditional intensity functions of event histories from a context defined by sets of event sequences.<n>Our amortized approach matches the performance of specialized models on next-event prediction across common benchmark datasets.
• Score: 12.037629169675938
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamical rules and predict future events. Current neural network approaches to MTPP inference rely on training separate, specialized models for each target system. We pursue a radically different approach: drawing on amortized inference and in-context learning, we pretrain a deep neural network to infer, in-context, the conditional intensity functions of event histories from a context defined by sets of event sequences. Pretraining is performed on a large synthetic dataset of MTPPs sampled from a broad distribution of Hawkes processes. Once pretrained, our Foundation Inference Model for Point Processes (FIM-PP) can estimate MTPPs from real-world data without any additional training, or be rapidly finetuned to target systems. Experiments show that this amortized approach matches the performance of specialized models on next-event prediction across common benchmark datasets. Our pretrained model, repository and tutorials will soon be available online

Related papers

• Transferable Post-training via Inverse Value Learning [83.75002867411263]
  We propose modeling changes at the logits level during post-training using a separate neural network (i.e., the value network)<n>After training this network on a small base model using demonstrations, this network can be seamlessly integrated with other pre-trained models during inference.<n>We demonstrate that the resulting value network has broad transferability across pre-trained models of different parameter sizes.
  arXiv  Detail & Related papers  (2024-10-28T13:48:43Z)
• EventFlow: Forecasting Temporal Point Processes with Flow Matching [12.976042923229466]
  In machine learning it is common to model temporal point processes in an autoregressive fashion using a neural network.<n>We propose EventFlow, a non-autoregressive generative model for temporal point processes.
  arXiv  Detail & Related papers  (2024-10-09T20:57:00Z)
• Cumulative Distribution Function based General Temporal Point Processes [49.758080415846884]
  CuFun model represents a novel approach to TPPs that revolves around the Cumulative Distribution Function (CDF) Our approach addresses several critical issues inherent in traditional TPP modeling. Our contributions encompass the introduction of a pioneering CDF-based TPP model, the development of a methodology for incorporating past event information into future event prediction.
  arXiv  Detail & Related papers  (2024-02-01T07:21:30Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Interacting Diffusion Processes for Event Sequence Forecasting [20.380620709345898]
  We introduce a novel approach that incorporates a diffusion generative model. The model facilitates sequence-to-sequence prediction, allowing multi-step predictions based on historical event sequences. We demonstrate that our proposal outperforms state-of-the-art baselines for long-horizon forecasting of TPP.
  arXiv  Detail & Related papers  (2023-10-26T22:17:25Z)
• Enhancing Asynchronous Time Series Forecasting with Contrastive Relational Inference [21.51753838306655]
  Temporal point processes(TPPs) are the standard method for modeling such. Existing TPP models have focused on the conditional distribution of future events instead of explicitly modeling event interactions, imposing challenges for event predictions. We propose a novel approach that leverages a Neural Inference (NRI) to learn a graph that infers interactions while simultaneously learning dynamics patterns from observational data.
  arXiv  Detail & Related papers  (2023-09-06T09:47:03Z)
• Continuous-time convolutions model of event sequences [46.3471121117337]
  Event sequences are non-uniform and sparse, making traditional models unsuitable. We propose COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time. COTIC outperforms existing models in predicting the next event time and type, achieving an average rank of 1.5 compared to 3.714 for the nearest competitor.
  arXiv  Detail & Related papers  (2023-02-13T10:34:51Z)
• Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information [77.80071279597665]
  We propose an all-in-one single-stage pre-training approach, named Maximizing Multi-modal Mutual Information Pre-training (M3I Pre-training) Our approach achieves better performance than previous pre-training methods on various vision benchmarks, including ImageNet classification, object detection, LVIS long-tailed object detection, and ADE20k semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-17T18:59:49Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Modeling Continuous Time Sequences with Intermittent Observations using Marked Temporal Point Processes [25.074394338483575]
  A large fraction of data generated via human activities can be represented as a sequence of events over a continuous-time. Deep learning models over these continuous-time event sequences is a non-trivial task. In this work, we provide a novel unsupervised model and inference method for learning MTPP in presence of event sequences with missing events.
  arXiv  Detail & Related papers  (2022-06-23T18:23:20Z)
• A Multi-Channel Neural Graphical Event Model with Negative Evidence [76.51278722190607]
  Event datasets are sequences of events of various types occurring irregularly over the time-line. We propose a non-parametric deep neural network approach in order to estimate the underlying intensity functions.
  arXiv  Detail & Related papers  (2020-02-21T23:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.