Using Differentiable Programming for Flexible Statistical Modeling

• URL: http://arxiv.org/abs/2012.05722v1
• Date: Mon, 7 Dec 2020 12:33:49 GMT
• Title: Using Differentiable Programming for Flexible Statistical Modeling
• Authors: Maren Hackenberg, Marlon Grodd, Clemens Kreutz, Martina Fischer, Janina Esins, Linus Grabenhenrich, Christian Karagiannidis, Harald Binder
• Abstract summary: Differentiable programming has recently received much interest as a paradigm that facilitates taking gradients of computer programs. We show how differentiable programming can enable simple gradient-based optimization of a model by automatic differentiation. This allowed us to quickly prototype a model under time pressure that outperforms simpler benchmark models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differentiable programming has recently received much interest as a paradigm that facilitates taking gradients of computer programs. While the corresponding flexible gradient-based optimization approaches so far have been used predominantly for deep learning or enriching the latter with modeling components, we want to demonstrate that they can also be useful for statistical modeling per se, e.g., for quick prototyping when classical maximum likelihood approaches are challenging or not feasible. In an application from a COVID-19 setting, we utilize differentiable programming to quickly build and optimize a flexible prediction model adapted to the data quality challenges at hand. Specifically, we develop a regression model, inspired by delay differential equations, that can bridge temporal gaps of observations in the central German registry of COVID-19 intensive care cases for predicting future demand. With this exemplary modeling challenge, we illustrate how differentiable programming can enable simple gradient-based optimization of the model by automatic differentiation. This allowed us to quickly prototype a model under time pressure that outperforms simpler benchmark models. We thus exemplify the potential of differentiable programming also outside deep learning applications, to provide more options for flexible applied statistical modeling.

Related papers

• Modern, Efficient, and Differentiable Transport Equation Models using JAX: Applications to Population Balance Equations [0.6990493129893112]
  Population balance equation (PBE) models have potential to automate many engineering processes. In the pharmaceutical sector, crystallization model-based design can contribute to shortening excessive drug development timelines.
  arXiv  Detail & Related papers  (2024-11-01T17:07:41Z)
• Towards Learning Stochastic Population Models by Gradient Descent [0.0]
  We show that simultaneous estimation of parameters and structure poses major challenges for optimization procedures. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty.
  arXiv  Detail & Related papers  (2024-04-10T14:38:58Z)
• Evaluation of Differentially Constrained Motion Models for Graph-Based Trajectory Prediction [1.1947990549568765]
  This research investigates the performance of various motion models in combination with numerical solvers for the prediction task. The study shows that simpler models, such as low-order integrator models, are preferred over more complex, e.g., kinematic models, to achieve accurate predictions.
  arXiv  Detail & Related papers  (2023-04-11T10:15:20Z)
• Planning with Diffusion for Flexible Behavior Synthesis [125.24438991142573]
  We consider what it would look like to fold as much of the trajectory optimization pipeline as possible into the modeling problem. The core of our technical approach lies in a diffusion probabilistic model that plans by iteratively denoising trajectories.
  arXiv  Detail & Related papers  (2022-05-20T07:02:03Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Learning summary features of time series for likelihood free inference [93.08098361687722]
  We present a data-driven strategy for automatically learning summary features from time series data. Our results indicate that learning summary features from data can compete and even outperform LFI methods based on hand-crafted values.
  arXiv  Detail & Related papers  (2020-12-04T19:21:37Z)
• Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration [130.89746032163106]
  We propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data. We show that the energy function and sampler can be trained efficiently via a new variational form of power iteration. We present an energy model guided fuzzer for software testing that achieves comparable performance to well engineered fuzzing engines like libfuzzer.
  arXiv  Detail & Related papers  (2020-11-10T19:31:29Z)
• Conditional Generative Modeling via Learning the Latent Space [54.620761775441046]
  We propose a novel framework for conditional generation in multimodal spaces. It uses latent variables to model generalizable learning patterns. At inference, the latent variables are optimized to find optimal solutions corresponding to multiple output modes.
  arXiv  Detail & Related papers  (2020-10-07T03:11:34Z)
• Experimental Design for Overparameterized Learning with Application to Single Shot Deep Active Learning [5.141687309207561]
  Modern machine learning models are trained on large amounts of labeled data. Access to large volumes of labeled data is often limited or expensive. We propose a new design strategy for curating the training set.
  arXiv  Detail & Related papers  (2020-09-27T11:27:49Z)
• Maximum Entropy Model Rollouts: Fast Model Based Policy Optimization without Compounding Errors [10.906666680425754]
  We propose a Dyna-style model-based reinforcement learning algorithm, which we called Maximum Entropy Model Rollouts (MEMR) To eliminate the compounding errors, we only use our model to generate single-step rollouts.
  arXiv  Detail & Related papers  (2020-06-08T21:38:15Z)
• Hybrid modeling: Applications in real-time diagnosis [64.5040763067757]
  We outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models. We are using such models for real-time diagnosis applications.
  arXiv  Detail & Related papers  (2020-03-04T00:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.