Dynamic sparsity on dynamic regression models

• URL: http://arxiv.org/abs/2009.14131v1
• Date: Tue, 29 Sep 2020 16:26:08 GMT
• Title: Dynamic sparsity on dynamic regression models
• Authors: Paloma W. Uribe and Hedibert F. Lopes
• Abstract summary: We consider variable selection and shrinkage for the Gaussian dynamic linear regression within a Bayesian framework. We propose a novel method that allows for time-varying sparsity, based on an extension of spike-and-slab priors for dynamic models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the present work, we consider variable selection and shrinkage for the Gaussian dynamic linear regression within a Bayesian framework. In particular, we propose a novel method that allows for time-varying sparsity, based on an extension of spike-and-slab priors for dynamic models. This is done by assigning appropriate Markov switching priors for the time-varying coefficients' variances, extending the previous work of Ishwaran and Rao (2005). Furthermore, we investigate different priors, including the common Inverted gamma prior for the process variances, and other mixture prior distributions such as Gamma priors for both the spike and the slab, which leads to a mixture of Normal-Gammas priors (Griffin ad Brown, 2010) for the coefficients. In this sense, our prior can be view as a dynamic variable selection prior which induces either smoothness (through the slab) or shrinkage towards zero (through the spike) at each time point. The MCMC method used for posterior computation uses Markov latent variables that can assume binary regimes at each time point to generate the coefficients' variances. In that way, our model is a dynamic mixture model, thus, we could use the algorithm of Gerlach et al (2000) to generate the latent processes without conditioning on the states. Finally, our approach is exemplified through simulated examples and a real data application.

Related papers

• An Elementary Approach to Scheduling in Generative Diffusion Models [55.171367482496755]
  An elementary approach to characterizing the impact of noise scheduling and time discretization in generative diffusion models is developed.<n> Experiments across different datasets and pretrained models demonstrate that the time discretization strategy selected by our approach consistently outperforms baseline and search-based strategies.
  arXiv  Detail & Related papers  (2026-01-20T05:06:26Z)
• HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models [1.949927790632678]
  This paper builds on the log transform known as the Cole-Hopf transform in Brownian motion contexts. We develop a new algorithm, named the HJ-sampler, for inference for the inverse problem of a differential equation with given terminal observations.
  arXiv  Detail & Related papers  (2024-09-15T05:30:54Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces novel deep dynamical models designed to represent continuous-time sequences. We train the model using maximum likelihood estimation with Markov chain Monte Carlo. Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
  We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Fusion of Gaussian Processes Predictions with Monte Carlo Sampling [61.31380086717422]
  In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and integrate their outcomes.
  arXiv  Detail & Related papers  (2024-03-03T04:21:21Z)
• Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations [0.0]
  Variational mixtures with full-covariance structures suffer from a quadratic growth due to variational parameters with the number of parameters. We propose a method for constructing an initial Gaussian model approximation that can be used to warm-start variational inference.
  arXiv  Detail & Related papers  (2023-07-12T19:30:04Z)
• A Class of Two-Timescale Stochastic EM Algorithms for Nonconvex Latent Variable Models [21.13011760066456]
  The Expectation-Maximization (EM) algorithm is a popular choice for learning variable models. In this paper, we propose a general class of methods called Two-Time Methods.
  arXiv  Detail & Related papers  (2022-03-18T22:46:34Z)
• Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods [35.24886589614034]
  We consider quadratic definite Ising models on the hypercube with a general interaction $J$. Our general result implies the first time sampling algorithms for low-rank Ising models.
  arXiv  Detail & Related papers  (2022-02-17T21:43:50Z)
• Time varying regression with hidden linear dynamics [74.9914602730208]
  We revisit a model for time-varying linear regression that assumes the unknown parameters evolve according to a linear dynamical system. Counterintuitively, we show that when the underlying dynamics are stable the parameters of this model can be estimated from data by combining just two ordinary least squares estimates.
  arXiv  Detail & Related papers  (2021-12-29T23:37:06Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Estimation of Switched Markov Polynomial NARX models [75.91002178647165]
  We identify a class of models for hybrid dynamical systems characterized by nonlinear autoregressive (NARX) components. The proposed approach is demonstrated on a SMNARX problem composed by three nonlinear sub-models with specific regressors.
  arXiv  Detail & Related papers  (2020-09-29T15:00:47Z)
• On the Discrepancy between Density Estimation and Sequence Generation [92.70116082182076]
  log-likelihood is highly correlated with BLEU when we consider models within the same family. We observe no correlation between rankings of models across different families.
  arXiv  Detail & Related papers  (2020-02-17T20:13:35Z)
• Variational Autoencoders with Riemannian Brownian Motion Priors [4.8461049669050915]
  Variational Autoencoders (VAEs) represent the given data in a low-dimensional latent space, which is generally assumed to be Euclidean. Recent work has shown that this prior has a detrimental effect on model capacity, leading to subpar performance. We propose an efficient inference scheme that does not rely on the unknown normalizing factor of this prior.
  arXiv  Detail & Related papers  (2020-02-12T20:35:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.