A Unified Descriptive-Complexity Framework for Model Selection under Correlated Designs

• URL: http://arxiv.org/abs/2608.26618v1
• Date: Thu, 27 Aug 2026 05:02:14 GMT
• Title: A Unified Descriptive-Complexity Framework for Model Selection under Correlated Designs
• Abstract summary: We propose a Descriptive-Complexity Information Criterion (DCIC) that regularizes large candidate model collections through Kraft-admissible code lengths.<n>We establish selection consistency through approximation-error separation without relying on RIP-type conditions.<n>This extension yields class-size-model recovery under suitable identifiability conditions and risk adaptation across classes.
• Score: 8.856886974515467
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model selection becomes particularly challenging under strong predictor dependence and model-class uncertainty, especially when there are exponentially many models. We propose a Descriptive-Complexity Information Criterion (DCIC) that regularizes large candidate model collections through Kraft-admissible code lengths. Under sub-Weibull noise, we establish selection consistency through approximation-error separation without relying on RIP-type conditions, together with nonasymptotic oracle risk bounds that remain valid under model misspecification. The same coding principle places heterogeneous classes on a common complexity scale at a small additional class-identification cost. This extension yields class--model recovery under suitable identifiability conditions and risk adaptation across classes. We further develop a complexity-guided search path that makes the computation--statistics trade-off explicit. Large penalties yield polynomial-size retained search regions with high probability, whereas smaller penalties sharpen the oracle risk benchmark. Numerical experiments illustrate stable support recovery and favorable estimation performance under strong dependence and model-class uncertainty.

Related papers

• Behavioural Signatures of Risk-Sensitive Decision-Making in Large Language Models [121.70781851874035]
  Large language models (LLMs) are increasingly used in decision support.<n>It is important to understand whether their choices under uncertainty exhibit stable and interpretable behavioural regularities.<n>Here we examine this question using a controlled multi-model framework based on no-limit Texas Hold'em.
  arXiv  Detail & Related papers  (2026-07-11T10:33:39Z)
• Covariate-assisted Grade of Membership Models via Shared Latent Geometry [0.7939348535496568]
  The grade of membership model is a flexible latent variable model for analyzing multivariate categorical data through individual-level mixed membership scores.<n>Traditional approaches to incorporating auxiliary covariates typically rely on fully specified joint likelihoods, which are computationally intensive and sensitive to misspecification.<n>We introduce a covariate-assisted grade of membership model that integrates response and covariate information by exploiting their shared low-rank simplex geometry.
  arXiv  Detail & Related papers  (2026-01-24T02:30:36Z)
• Model Correlation Detection via Random Selection Probing [62.093777777813756]
  Existing similarity-based methods require access to model parameters or produce scores without thresholds.<n>We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test.<n>RSP produces rigorous p-values that quantify evidence of correlation.
  arXiv  Detail & Related papers  (2025-09-29T01:40:26Z)
• Generalized Criterion for Identifiability of Additive Noise Models Using Majorization [7.448620208767376]
  We introduce a novel identifiability criterion for directed acyclic graph (DAG) models. We demonstrate that this criterion extends and generalizes existing identifiability criteria. We present a new algorithm for learning a topological ordering of variables.
  arXiv  Detail & Related papers  (2024-04-08T02:18:57Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• Optimal Multi-Distribution Learning [88.3008613028333]
  Multi-distribution learning seeks to learn a shared model that minimizes the worst-case risk across $k$ distinct data distributions.<n>We propose a novel algorithm that yields an varepsilon-optimal randomized hypothesis with a sample complexity on the order of (d+k)/varepsilon2.
  arXiv  Detail & Related papers  (2023-12-08T16:06:29Z)
• Factor-Augmented Regularized Model for Hazard Regression [1.8021287677546953]
  We propose a new model, Factor-Augmented Regularized Model for Hazard Regression (FarmHazard), to perform model selection in high-dimensional data. We prove model selection consistency and estimation consistency under mild conditions. We also develop a factor-augmented variable screening procedure to deal with strong correlations in ultra-high dimensional problems.
  arXiv  Detail & Related papers  (2022-10-03T16:35:33Z)
• On the Complexity of Adversarial Decision Making [101.14158787665252]
  We show that the Decision-Estimation Coefficient is necessary and sufficient to obtain low regret for adversarial decision making. We provide new structural results that connect the Decision-Estimation Coefficient to variants of other well-known complexity measures.
  arXiv  Detail & Related papers  (2022-06-27T06:20:37Z)
• Sample Complexity of Robust Reinforcement Learning with a Generative Model [0.0]
  We propose a model-based reinforcement learning (RL) algorithm for learning an $epsilon$-optimal robust policy. We consider three different forms of uncertainty sets, characterized by the total variation distance, chi-square divergence, and KL divergence. In addition to the sample complexity results, we also present a formal analytical argument on the benefit of using robust policies.
  arXiv  Detail & Related papers  (2021-12-02T18:55:51Z)
• A non-asymptotic model selection in block-diagonal mixture of polynomial experts models [1.491109220586182]
  We introduce a penalized maximum likelihood selection criterion to estimate the unknown conditional density of a regression model. We provide a strong theoretical guarantee, including a finite-sample oracle satisfied by the penalized maximum likelihood with a Jensen-Kullback-Leibler type loss.
  arXiv  Detail & Related papers  (2021-04-18T21:32:20Z)
• Characterizing Fairness Over the Set of Good Models Under Selective Labels [69.64662540443162]
  We develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance. We provide tractable algorithms to compute the range of attainable group-level predictive disparities. We extend our framework to address the empirically relevant challenge of selectively labelled data.
  arXiv  Detail & Related papers  (2021-01-02T02:11:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.