How Inverse Conditional Flows Can Serve as a Substitute for Distributional Regression

• URL: http://arxiv.org/abs/2405.05429v3
• Date: Wed, 10 Jul 2024 08:47:05 GMT
• Title: How Inverse Conditional Flows Can Serve as a Substitute for Distributional Regression
• Authors: Lucas Kook, Chris Kolb, Philipp Schiele, Daniel Dold, Marcel Arpogaus, Cornelius Fritz, Philipp F. Baumann, Philipp Kopper, Tobias Pielok, Emilio Dorigatti, David Rügamer,
• Abstract summary: We propose a framework for distributional regression using inverse flow transformations (DRIFT) DRIFT covers both interpretable statistical models and flexible neural networks opening up new avenues in both statistical modeling and deep learning.
• Score: 2.9873759776815527
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural network representations of simple models, such as linear regression, are being studied increasingly to better understand the underlying principles of deep learning algorithms. However, neural representations of distributional regression models, such as the Cox model, have received little attention so far. We close this gap by proposing a framework for distributional regression using inverse flow transformations (DRIFT), which includes neural representations of the aforementioned models. We empirically demonstrate that the neural representations of models in DRIFT can serve as a substitute for their classical statistical counterparts in several applications involving continuous, ordered, time-series, and survival outcomes. We confirm that models in DRIFT empirically match the performance of several statistical methods in terms of estimation of partial effects, prediction, and aleatoric uncertainty quantification. DRIFT covers both interpretable statistical models and flexible neural networks opening up new avenues in both statistical modeling and deep learning.

Related papers

• Neural Frailty Machine: Beyond proportional hazard assumption in neural survival regressions [30.018173329118184]
  We present neural frailty machine (NFM), a powerful and flexible neural modeling framework for survival regressions. Two concrete models are derived under the framework that extends neural proportional hazard models and non hazard regression models. We conduct experimental evaluations over $6$ benchmark datasets of different scales, showing that the proposed NFM models outperform state-of-the-art survival models in terms of predictive performance.
  arXiv  Detail & Related papers  (2023-03-18T08:15:15Z)
• A predictive physics-aware hybrid reduced order model for reacting flows [65.73506571113623]
  A new hybrid predictive Reduced Order Model (ROM) is proposed to solve reacting flow problems. The number of degrees of freedom is reduced from thousands of temporal points to a few POD modes with their corresponding temporal coefficients. Two different deep learning architectures have been tested to predict the temporal coefficients.
  arXiv  Detail & Related papers  (2023-01-24T08:39:20Z)
• Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST) [1.8047694351309207]
  We develop a statistical framework for model predictions based on transfer learning, called RECaST. We mathematically and empirically demonstrate the validity of our RECaST approach for transfer learning between linear models. We examine our method's performance in a simulation study and in an application to real hospital data.
  arXiv  Detail & Related papers  (2022-11-29T19:39:47Z)
• Learning Stochastic Dynamics with Statistics-Informed Neural Network [0.4297070083645049]
  We introduce a machine-learning framework named statistics-informed neural network (SINN) for learning dynamics from data. We devise mechanisms for training the neural network model to reproduce the correct emphstatistical behavior of a target process. We show that the obtained reduced-order model can be trained on temporally coarse-grained data and hence is well suited for rare-event simulations.
  arXiv  Detail & Related papers  (2022-02-24T18:21:01Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Regularized Sequential Latent Variable Models with Adversarial Neural Networks [33.74611654607262]
  We will present different ways of using high level latent random variables in RNN to model the variability in the sequential data. We will explore possible ways of using adversarial method to train a variational RNN model.
  arXiv  Detail & Related papers  (2021-08-10T08:05:14Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Sparsely constrained neural networks for model discovery of PDEs [0.0]
  We present a modular framework that determines the sparsity pattern of a deep-learning based surrogate using any sparse regression technique. We show how a different network architecture and sparsity estimator improve model discovery accuracy and convergence on several benchmark examples.
  arXiv  Detail & Related papers  (2020-11-09T11:02:40Z)
• A Bayesian Perspective on Training Speed and Model Selection [51.15664724311443]
  We show that a measure of a model's training speed can be used to estimate its marginal likelihood. We verify our results in model selection tasks for linear models and for the infinite-width limit of deep neural networks. Our results suggest a promising new direction towards explaining why neural networks trained with gradient descent are biased towards functions that generalize well.
  arXiv  Detail & Related papers  (2020-10-27T17:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.