Fitting Sparse Markov Models to Categorical Time Series Using Convex Clustering

• URL: http://arxiv.org/abs/2202.05485v2
• Date: Wed, 02 Jul 2025 01:20:24 GMT
• Title: Fitting Sparse Markov Models to Categorical Time Series Using Convex Clustering
• Authors: Tuhin Majumder, Soumendra Lahiri, Donald Martin,
• Abstract summary: A parsimonious modeling approach is given by Sparse Markov Models (SMMs)<n>We develop an elegant method of fitting SMMs based on convex clustering and regularization.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Higher-order Markov chains are frequently used to model categorical time series. However, a major problem with fitting such models is the exponentially growing number of parameters in the model order. A popular approach to parsimonious modeling is to use a Variable Length Markov Chain (VLMC), which determines relevant contexts (recent pasts) of variable orders and forms a context tree. A more general parsimonious modeling approach is given by Sparse Markov Models (SMMs), where all possible histories of order $m$ are partitioned such that the transition probability vectors are identical for the histories belonging to any particular group. In this paper, we develop an elegant method of fitting SMMs based on convex clustering and regularization. The regularization parameter is selected using the BIC criterion. Theoretical results establish model selection consistency of our method for large sample size. Extensive simulation results under different set-ups are presented to study finite sample performance of the method. Real data analysis on modelling and classifying disease sub-types demonstrates the applicability of our method as well.

Related papers

• AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models with Unsupervised Coefficient Optimization [86.8133939108057]
  We propose AdaMMS, a novel model merging method tailored for heterogeneous MLLMs. Our method tackles the challenges in three steps: mapping, merging and searching. As the first model merging method capable of merging heterogeneous MLLMs without labeled data, AdaMMS outperforms previous model merging methods on various vision-language benchmarks.
  arXiv  Detail & Related papers  (2025-03-31T05:13:02Z)
• On the Expressiveness and Length Generalization of Selective State-Space Models on Regular Languages [56.22289522687125]
  Selective state-space models (SSMs) are an emerging alternative to the Transformer. We analyze their expressiveness and length generalization performance on regular language tasks. We introduce the Selective Dense State-Space Model (SD-SSM), the first selective SSM that exhibits perfect length generalization.
  arXiv  Detail & Related papers  (2024-12-26T20:53:04Z)
• Ensemble Methods for Sequence Classification with Hidden Markov Models [8.241486511994202]
  We present a lightweight approach to sequence classification using Ensemble Methods for Hidden Markov Models (HMMs) HMMs offer significant advantages in scenarios with imbalanced or smaller datasets due to their simplicity, interpretability, and efficiency. Our ensemble-based scoring method enables the comparison of sequences of any length and improves performance on imbalanced datasets.
  arXiv  Detail & Related papers  (2024-09-11T20:59:32Z)
• Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation [60.493180081319785]
  We propose a systematic way to estimate the intrinsic capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step. Our work provides a comprehensive comparison between existing truncation sampling methods, as well as their recommended parameters as a guideline for users.
  arXiv  Detail & Related papers  (2024-08-24T14:14:32Z)
• Random Models for Fuzzy Clustering Similarity Measures [0.0]
  The Adjusted Rand Index (ARI) is a widely used method for comparing hard clusterings. We propose a single framework for computing the ARI with three random models that are intuitive and explainable for both hard and fuzzy clusterings.
  arXiv  Detail & Related papers  (2023-12-16T00:07:04Z)
• Variable Clustering via Distributionally Robust Nodewise Regression [7.289979396903827]
  We study a multi-factor block model for variable clustering and connect it to the regularized subspace clustering by formulating a distributionally robust version of the nodewise regression. We derive a convex relaxation, provide guidance on selecting the size of the robust region, and hence the regularization weighting parameter, based on the data, and propose an ADMM algorithm for implementation.
  arXiv  Detail & Related papers  (2022-12-15T16:23:25Z)
• Optimization of Annealed Importance Sampling Hyperparameters [77.34726150561087]
  Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models. We present a parameteric AIS process with flexible intermediary distributions and optimize the bridging distributions to use fewer number of steps for sampling. We assess the performance of our optimized AIS for marginal likelihood estimation of deep generative models and compare it to other estimators.
  arXiv  Detail & Related papers  (2022-09-27T07:58:25Z)
• Time Series Clustering with an EM algorithm for Mixtures of Linear Gaussian State Space Models [0.0]
  We propose a novel model-based time series clustering method with mixtures of linear Gaussian state space models. The proposed method uses a new expectation-maximization algorithm for the mixture model to estimate the model parameters. Experiments on a simulated dataset demonstrate the effectiveness of the method in clustering, parameter estimation, and model selection.
  arXiv  Detail & Related papers  (2022-08-25T07:41:23Z)
• Personalized Federated Learning via Convex Clustering [72.15857783681658]
  We propose a family of algorithms for personalized federated learning with locally convex user costs. The proposed framework is based on a generalization of convex clustering in which the differences between different users' models are penalized.
  arXiv  Detail & Related papers  (2022-02-01T19:25:31Z)
• Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability [25.543231171094384]
  In pursuit of explainability, we develop generative models for sequential data. We combine modern neural networks (normalizing flows) and traditional generative models (hidden Markov models - HMMs) The proposed generative models can compute likelihood of a data and hence directly suitable for maximum-likelihood (ML) classification approach.
  arXiv  Detail & Related papers  (2021-07-01T20:10:55Z)
• Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept [56.46135010588918]
  We prove that the widely used class of RNN-Transducer models and segmental models (direct HMM) are equivalent. It is shown that blank probabilities translate into segment length probabilities and vice versa.
  arXiv  Detail & Related papers  (2021-04-13T11:20:48Z)
• Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows [25.543231171094384]
  We use a generative model that combines the state transitions of a hidden Markov model (HMM) and the neural network based probability distributions for the hidden states of the HMM. We verify the improved robustness of NMM-HMM classifiers in an application to speech recognition.
  arXiv  Detail & Related papers  (2021-02-15T00:40:30Z)
• Community Detection in the Stochastic Block Model by Mixed Integer Programming [3.8073142980733]
  Degree-Corrected Block Model (DCSBM) is a popular model to generate random graphs with community structure given an expected degree sequence. Standard approach of community detection based on the DCSBM is to search for the model parameters that are the most likely to have produced the observed network data through maximum likelihood estimation (MLE) We present mathematical programming formulations and exact solution methods that can provably find the model parameters and community assignments of maximum likelihood given an observed graph.
  arXiv  Detail & Related papers  (2021-01-26T22:04:40Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• Generalized Matrix Factorization: efficient algorithms for fitting generalized linear latent variable models to large data arrays [62.997667081978825]
  Generalized Linear Latent Variable models (GLLVMs) generalize such factor models to non-Gaussian responses. Current algorithms for estimating model parameters in GLLVMs require intensive computation and do not scale to large datasets. We propose a new approach for fitting GLLVMs to high-dimensional datasets, based on approximating the model using penalized quasi-likelihood.
  arXiv  Detail & Related papers  (2020-10-06T04:28:19Z)
• Semi-nonparametric Latent Class Choice Model with a Flexible Class Membership Component: A Mixture Model Approach [6.509758931804479]
  The proposed model formulates the latent classes using mixture models as an alternative approach to the traditional random utility specification. Results show that mixture models improve the overall performance of latent class choice models.
  arXiv  Detail & Related papers  (2020-07-06T13:19:26Z)
• Slice Sampling for General Completely Random Measures [74.24975039689893]
  We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables. The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
  arXiv  Detail & Related papers  (2020-06-24T17:53:53Z)
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology [66.06153115971732]
  We present a method for quantifying disentanglement that only uses the generative model. We empirically evaluate several state-of-the-art models across multiple datasets.
  arXiv  Detail & Related papers  (2020-06-05T20:54:11Z)
• On the Discrepancy between Density Estimation and Sequence Generation [92.70116082182076]
  log-likelihood is highly correlated with BLEU when we consider models within the same family. We observe no correlation between rankings of models across different families.
  arXiv  Detail & Related papers  (2020-02-17T20:13:35Z)
• Clustering Binary Data by Application of Combinatorial Optimization Heuristics [52.77024349608834]
  We study clustering methods for binary data, first defining aggregation criteria that measure the compactness of clusters. Five new and original methods are introduced, using neighborhoods and population behavior optimization metaheuristics. From a set of 16 data tables generated by a quasi-Monte Carlo experiment, a comparison is performed for one of the aggregations using L1 dissimilarity, with hierarchical clustering, and a version of k-means: partitioning around medoids or PAM.
  arXiv  Detail & Related papers  (2020-01-06T23:33:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.