Foundation Inference Models for Markov Jump Processes

• URL: http://arxiv.org/abs/2406.06419v3
• Date: Mon, 03 Mar 2025 11:26:33 GMT
• Title: Foundation Inference Models for Markov Jump Processes
• Authors: David Berghaus, Kostadin Cvejoski, Patrick Seifner, Cesar Ojeda, Ramses J. Sanchez,
• Abstract summary: We introduce a methodology for zero-shot inference of Markov jump processes (MJPs) on bounded state spaces.<n>One and the same (pretrained) model can infer, in a zero-shot fashion, hidden MJPs evolving in state spaces of different dimensionalities.
• Score: 3.005912045854039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Markov jump processes are continuous-time stochastic processes which describe dynamical systems evolving in discrete state spaces. These processes find wide application in the natural sciences and machine learning, but their inference is known to be far from trivial. In this work we introduce a methodology for zero-shot inference of Markov jump processes (MJPs), on bounded state spaces, from noisy and sparse observations, which consists of two components. First, a broad probability distribution over families of MJPs, as well as over possible observation times and noise mechanisms, with which we simulate a synthetic dataset of hidden MJPs and their noisy observation process. Second, a neural network model that processes subsets of the simulated observations, and that is trained to output the initial condition and rate matrix of the target MJP in a supervised way. We empirically demonstrate that one and the same (pretrained) model can infer, in a zero-shot fashion, hidden MJPs evolving in state spaces of different dimensionalities. Specifically, we infer MJPs which describe (i) discrete flashing ratchet systems, which are a type of Brownian motors, and the conformational dynamics in (ii) molecular simulations, (iii) experimental ion channel data and (iv) simple protein folding models. What is more, we show that our model performs on par with state-of-the-art models which are finetuned to the target datasets.

Related papers

• MarS-FM: Generative Modeling of Molecular Dynamics via Markov State Models [4.818682161738301]
  We introduce a new class of generative models, MSM Emulators, which learn to sample transitions across discrete states defined by an underlying Markov State Model (MSM)<n>Our evaluation spans protein domains with significant chemical and structural diversity, including unfolding events, and enforces strict sequence dissimilarity between training and test sets to assess generalization.
  arXiv  Detail & Related papers  (2025-09-29T13:42:25Z)
• Efficient Deconvolution in Populational Inverse Problems [6.1048542188987085]
  This work is focussed on the inversion task of inferring the distribution over parameters of interest leading to multiple sets of observations.<n>The potential to solve such distributional inversion problems is driven by increasing availability of data, but a major roadblock is blind deconvolution.<n>We propose a methodology leveraging large data sets of observations, collected from different instantiations of the same physical processes, to simultaneously deconvolve the data corrupting noise distribution.
  arXiv  Detail & Related papers  (2025-05-26T11:25:46Z)
• [MASK] is All You Need [28.90875822599164]
  We propose using discrete-state models to connect Masked Generative and Non-autoregressive Diffusion models. By leveraging [MASK] in discrete-state models, we can bridge Masked Generative and Non-autoregressive Diffusion models.
  arXiv  Detail & Related papers  (2024-12-09T18:59:56Z)
• MING: A Functional Approach to Learning Molecular Generative Models [46.189683355768736]
  This paper introduces a novel paradigm for learning molecule generative models based on functional representations. We propose Molecular Implicit Neural Generation (MING), a diffusion-based model that learns molecular distributions in function space.
  arXiv  Detail & Related papers  (2024-10-16T13:02:02Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Data-Driven Model Selections of Second-Order Particle Dynamics via Integrating Gaussian Processes with Low-Dimensional Interacting Structures [0.9821874476902972]
  We focus on the data-driven discovery of a general second-order particle-based model. We present applications to modeling two real-world fish motion datasets.
  arXiv  Detail & Related papers  (2023-11-01T23:45:15Z)
• Learning minimal representations of stochastic processes with variational autoencoders [52.99137594502433]
  We introduce an unsupervised machine learning approach to determine the minimal set of parameters required to describe a process. Our approach enables for the autonomous discovery of unknown parameters describing processes.
  arXiv  Detail & Related papers  (2023-07-21T14:25:06Z)
• Free-Form Variational Inference for Gaussian Process State-Space Models [21.644570034208506]
  We propose a new method for inference in Bayesian GPSSMs. Our method is based on freeform variational inference via inducing Hamiltonian Monte Carlo. We show that our approach can learn transition dynamics and latent states more accurately than competing methods.
  arXiv  Detail & Related papers  (2023-02-20T11:34:16Z)
• Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series [18.885471782270375]
  NCDSSM employs auxiliary variables to disentangle recognition from dynamics, thus requiring amortized inference only for the auxiliary variables. We propose three flexible parameterizations of the latent dynamics and an efficient training objective that marginalizes the dynamic states during inference. Empirical results on multiple benchmark datasets show improved imputation and forecasting performance of NCDSSM over existing models.
  arXiv  Detail & Related papers  (2023-01-26T18:45:04Z)
• Conditional Neural Processes for Molecules [0.0]
  Neural processes (NPs) are models for transfer learning with properties reminiscent of Gaussian Processes (GPs) This paper applies the conditional neural process (CNP) to DOCKSTRING, a dataset of docking scores for benchmarking ML models. CNPs show competitive performance in few-shot learning tasks relative to supervised learning baselines common in QSAR modelling, as well as an alternative model for transfer learning based on pre-training and refining neural network regressors.
  arXiv  Detail & Related papers  (2022-10-17T16:10:12Z)
• DriPP: Driven Point Processes to Model Stimuli Induced Patterns in M/EEG Signals [62.997667081978825]
  We develop a novel statistical point process model-called driven temporal point processes (DriPP) We derive a fast and principled expectation-maximization (EM) algorithm to estimate the parameters of this model. Results on standard MEG datasets demonstrate that our methodology reveals event-related neural responses.
  arXiv  Detail & Related papers  (2021-12-08T13:07:21Z)
• Likelihood-Free Inference in State-Space Models with Unknown Dynamics [71.94716503075645]
  We introduce a method for inferring and predicting latent states in state-space models where observations can only be simulated, and transition dynamics are unknown. We propose a way of doing likelihood-free inference (LFI) of states and state prediction with a limited number of simulations.
  arXiv  Detail & Related papers  (2021-11-02T12:33:42Z)
• On Contrastive Representations of Stochastic Processes [53.21653429290478]
  Learning representations of processes is an emerging problem in machine learning. We show that our methods are effective for learning representations of periodic functions, 3D objects and dynamical processes.
  arXiv  Detail & Related papers  (2021-06-18T11:00:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.