Model Selection Through Model Sorting

• URL: http://arxiv.org/abs/2409.09674v1
• Date: Sun, 15 Sep 2024 09:43:59 GMT
• Title: Model Selection Through Model Sorting
• Authors: Mohammad Ali Hajiani, Babak Seyfe,
• Abstract summary: We propose a model order selection method called nested empirical risk (NER) In the UCR data set, the NER method reduces the complexity of the classification of UCR datasets dramatically.
• Score: 1.534667887016089
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel approach to select the best model of the data. Based on the exclusive properties of the nested models, we find the most parsimonious model containing the risk minimizer predictor. We prove the existence of probable approximately correct (PAC) bounds on the difference of the minimum empirical risk of two successive nested models, called successive empirical excess risk (SEER). Based on these bounds, we propose a model order selection method called nested empirical risk (NER). By the sorted NER (S-NER) method to sort the models intelligently, the minimum risk decreases. We construct a test that predicts whether expanding the model decreases the minimum risk or not. With a high probability, the NER and S-NER choose the true model order and the most parsimonious model containing the risk minimizer predictor, respectively. We use S-NER model selection in the linear regression and show that, the S-NER method without any prior information can outperform the accuracy of feature sorting algorithms like orthogonal matching pursuit (OMP) that aided with prior knowledge of the true model order. Also, in the UCR data set, the NER method reduces the complexity of the classification of UCR datasets dramatically, with a negligible loss of accuracy.

Related papers

• Inverse Reinforcement Learning with Unknown Reward Model based on Structural Risk Minimization [9.44879308639364]
  Inverse reinforcement learning (IRL) usually assumes the model of the reward function is pre-specified and estimates the parameter only. A simplistic model is less likely to contain the real reward function, while a model with high complexity leads to substantial cost and risks overfitting. This paper addresses this trade-off by introducing the structural risk minimization (SRM) method from statistical learning.
  arXiv  Detail & Related papers  (2023-12-27T13:23:17Z)
• Learning Rich Rankings [7.940293148084844]
  We develop a contextual repeated selection (CRS) model to bring a natural multimodality and richness to the rankings space. We provide theoretical guarantees for maximum likelihood estimation under the model through structure-dependent tail risk and expected risk bounds. We also furnish the first tight bounds on the expected risk of maximum likelihood estimators for the multinomial logit (MNL) choice model and the Plackett-Luce (PL) ranking model.
  arXiv  Detail & Related papers  (2023-12-22T21:40:57Z)
• Random Models for Fuzzy Clustering Similarity Measures [0.0]
  The Adjusted Rand Index (ARI) is a widely used method for comparing hard clusterings. We propose a single framework for computing the ARI with three random models that are intuitive and explainable for both hard and fuzzy clusterings.
  arXiv  Detail & Related papers  (2023-12-16T00:07:04Z)
• Optimal Multi-Distribution Learning [88.3008613028333]
  Multi-distribution learning seeks to learn a shared model that minimizes the worst-case risk across $k$ distinct data distributions. We propose a novel algorithm that yields an varepsilon-optimal randomized hypothesis with a sample complexity on the order of (d+k)/varepsilon2.
  arXiv  Detail & Related papers  (2023-12-08T16:06:29Z)
• BRIO: Bringing Order to Abstractive Summarization [107.97378285293507]
  We propose a novel training paradigm which assumes a non-deterministic distribution. Our method achieves a new state-of-the-art result on the CNN/DailyMail (47.78 ROUGE-1) and XSum (49.07 ROUGE-1) datasets.
  arXiv  Detail & Related papers  (2022-03-31T05:19:38Z)
• Universal and data-adaptive algorithms for model selection in linear contextual bandits [52.47796554359261]
  We consider the simplest non-trivial instance of model-selection: distinguishing a simple multi-armed bandit problem from a linear contextual bandit problem. We introduce new algorithms that explore in a data-adaptive manner and provide guarantees of the form $mathcalO(dalpha T1- alpha)$. Our approach extends to model selection among nested linear contextual bandits under some additional assumptions.
  arXiv  Detail & Related papers  (2021-11-08T18:05:35Z)
• Meta-Model Structure Selection: Building Polynomial NARX Model for Regression and Classification [0.0]
  This work presents a new meta-heuristic approach to select the structure of NARX models for regression and classification problems. The robustness of the new algorithm is tested on several simulated and experimental system with different nonlinear characteristics.
  arXiv  Detail & Related papers  (2021-09-21T02:05:40Z)
• Modeling the Second Player in Distributionally Robust Optimization [90.25995710696425]
  We argue for the use of neural generative models to characterize the worst-case distribution. This approach poses a number of implementation and optimization challenges. We find that the proposed approach yields models that are more robust than comparable baselines.
  arXiv  Detail & Related papers  (2021-03-18T14:26:26Z)
• On the Minimal Error of Empirical Risk Minimization [90.09093901700754]
  We study the minimal error of the Empirical Risk Minimization (ERM) procedure in the task of regression. Our sharp lower bounds shed light on the possibility (or impossibility) of adapting to simplicity of the model generating the data.
  arXiv  Detail & Related papers  (2021-02-24T04:47:55Z)
• Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature [61.22680308681648]
  We show that global convergence is statistically intractable even for one-layer neural net bandit with a deterministic reward. For both nonlinear bandit and RL, the paper presents a model-based algorithm, Virtual Ascent with Online Model Learner (ViOL)
  arXiv  Detail & Related papers  (2021-02-08T12:41:56Z)
• On Statistical Efficiency in Learning [37.08000833961712]
  We address the challenge of model selection to strike a balance between model fitting and model complexity. We propose an online algorithm that sequentially expands the model complexity to enhance selection stability and reduce cost. Experimental studies show that the proposed method has desirable predictive power and significantly less computational cost than some popular methods.
  arXiv  Detail & Related papers  (2020-12-24T16:08:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.