On the Efficient Marginalization of Probabilistic Sequence Models

• URL: http://arxiv.org/abs/2403.04005v1
• Date: Wed, 6 Mar 2024 19:29:08 GMT
• Title: On the Efficient Marginalization of Probabilistic Sequence Models
• Authors: Alex Boyd
• Abstract summary: This dissertation focuses on using autoregressive models to answer complex probabilistic queries. We develop a class of novel and efficient approximation techniques for marginalization in sequential models that are model-agnostic.
• Score: 3.5897534810405403
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world data often exhibits sequential dependence, across diverse domains such as human behavior, medicine, finance, and climate modeling. Probabilistic methods capture the inherent uncertainty associated with prediction in these contexts, with autoregressive models being especially prominent. This dissertation focuses on using autoregressive models to answer complex probabilistic queries that go beyond single-step prediction, such as the timing of future events or the likelihood of a specific event occurring before another. In particular, we develop a broad class of novel and efficient approximation techniques for marginalization in sequential models that are model-agnostic. These techniques rely solely on access to and sampling from next-step conditional distributions of a pre-trained autoregressive model, including both traditional parametric models as well as more recent neural autoregressive models. Specific approaches are presented for discrete sequential models, for marked temporal point processes, and for stochastic jump processes, each tailored to a well-defined class of informative, long-range probabilistic queries.

Related papers

• ProGen: Revisiting Probabilistic Spatial-Temporal Time Series Forecasting from a Continuous Generative Perspective Using Stochastic Differential Equations [18.64802090861607]
  ProGen Pro provides a robust solution that effectively captures dependencies while managing uncertainty. Our experiments on four benchmark traffic datasets demonstrate that ProGen Pro outperforms state-of-the-art deterministic probabilistic models.
  arXiv  Detail & Related papers  (2024-11-02T14:37:30Z)
• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• Likelihood Based Inference in Fully and Partially Observed Exponential Family Graphical Models with Intractable Normalizing Constants [4.532043501030714]
  Probabilistic graphical models that encode an underlying Markov random field are fundamental building blocks of generative modeling. This paper is to demonstrate that full likelihood based analysis of these models is feasible in a computationally efficient manner.
  arXiv  Detail & Related papers  (2024-04-27T02:58:22Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Probabilistic Modeling for Sequences of Sets in Continuous-Time [14.423456635520084]
  We develop a general framework for modeling set-valued data in continuous-time. We also develop inference methods that can use such models to answer probabilistic queries.
  arXiv  Detail & Related papers  (2023-12-22T20:16:10Z)
• Interacting Diffusion Processes for Event Sequence Forecasting [20.380620709345898]
  We introduce a novel approach that incorporates a diffusion generative model. The model facilitates sequence-to-sequence prediction, allowing multi-step predictions based on historical event sequences. We demonstrate that our proposal outperforms state-of-the-art baselines for long-horizon forecasting of TPP.
  arXiv  Detail & Related papers  (2023-10-26T22:17:25Z)
• ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
  We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data. Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
  arXiv  Detail & Related papers  (2023-04-07T15:17:48Z)
• Stochastic Parameterizations: Better Modelling of Temporal Correlations using Probabilistic Machine Learning [1.5293427903448025]
  We show that by using a physically-informed recurrent neural network within a probabilistic framework, our model for the 96 atmospheric simulation is competitive. This is due to a superior ability to model temporal correlations compared to standard first-order autoregressive schemes. We evaluate across a number of metrics from the literature, but also discuss how the probabilistic metric of likelihood may be a unifying choice for future climate models.
  arXiv  Detail & Related papers  (2022-03-28T14:51:42Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.