Trajectory Flow Matching with Applications to Clinical Time Series Modeling

• URL: http://arxiv.org/abs/2410.21154v1
• Date: Mon, 28 Oct 2024 15:54:50 GMT
• Title: Trajectory Flow Matching with Applications to Clinical Time Series Modeling
• Authors: Xi Zhang, Yuan Pu, Yuki Kawamura, Andrew Loza, Yoshua Bengio, Dennis L. Shung, Alexander Tong,
• Abstract summary: Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
• Score: 77.58277281319253
• License:
• Abstract: Modeling stochastic and irregularly sampled time series is a challenging problem found in a wide range of applications, especially in medicine. Neural stochastic differential equations (Neural SDEs) are an attractive modeling technique for this problem, which parameterize the drift and diffusion terms of an SDE with neural networks. However, current algorithms for training Neural SDEs require backpropagation through the SDE dynamics, greatly limiting their scalability and stability. To address this, we propose Trajectory Flow Matching (TFM), which trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. TFM leverages the flow matching technique from generative modeling to model time series. In this work we first establish necessary conditions for TFM to learn time series data. Next, we present a reparameterization trick which improves training stability. Finally, we adapt TFM to the clinical time series setting, demonstrating improved performance on three clinical time series datasets both in terms of absolute performance and uncertainty prediction.

Related papers

• Physics-constrained coupled neural differential equations for one dimensional blood flow modeling [0.3749861135832073]
  Computational cardiovascular flow modeling plays a crucial role in understanding blood flow dynamics. Traditional 1D models based on finite element methods (FEM) often lack accuracy compared to 3D averaged solutions. This study introduces a novel physics-constrained machine learning technique that enhances the accuracy of 1D blood flow models.
  arXiv  Detail & Related papers  (2024-11-08T15:22:20Z)
• Equivariant Graph Neural Operator for Modeling 3D Dynamics [148.98826858078556]
  We propose Equivariant Graph Neural Operator (EGNO) to directly models dynamics as trajectories instead of just next-step prediction. EGNO explicitly learns the temporal evolution of 3D dynamics where we formulate the dynamics as a function over time and learn neural operators to approximate it. Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods.
  arXiv  Detail & Related papers  (2024-01-19T21:50:32Z)
• A Neural PDE Solver with Temporal Stencil Modeling [44.97241931708181]
  Recent Machine Learning (ML) models have shown new promises in capturing important dynamics in high-resolution signals. This study shows that significant information is often lost in the low-resolution down-sampled features. We propose a new approach, which combines the strengths of advanced time-series sequence modeling and state-of-the-art neural PDE solvers.
  arXiv  Detail & Related papers  (2023-02-16T06:13:01Z)
• Learning PDE Solution Operator for Continuous Modeling of Time-Series [1.39661494747879]
  This work presents a partial differential equation (PDE) based framework which improves the dynamics modeling capability. We propose a neural operator that can handle time continuously without requiring iterative operations or specific grids of temporal discretization. Our framework opens up a new way for a continuous representation of neural networks that can be readily adopted for real-world applications.
  arXiv  Detail & Related papers  (2023-02-02T03:47:52Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Neural ODE Processes [64.10282200111983]
  We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs. We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
  arXiv  Detail & Related papers  (2021-03-23T09:32:06Z)
• Accelerating Simulation of Stiff Nonlinear Systems using Continuous-Time Echo State Networks [1.1545092788508224]
  We present a data-driven method for generating surrogates of nonlinear ordinary differential equations with dynamics at widely separated timescales. We empirically demonstrate near-constant time performance using our CTESNs on a physically motivated scalable model of a heating system.
  arXiv  Detail & Related papers  (2020-10-07T17:40:06Z)
• Learning Continuous-Time Dynamics by Stochastic Differential Networks [32.63114111531396]
  We propose a flexible continuous-time recurrent neural network named Variational Differential Networks (VSDN) VSDN embeds the complicated dynamics of the sporadic time series by neural Differential Equations (SDE) We show that VSDNs outperform state-of-the-art continuous-time deep learning models and achieve remarkable performance on prediction and tasks for sporadic time series.
  arXiv  Detail & Related papers  (2020-06-11T01:40:34Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.