A Hybrid Tsallis-Polarization Impurity Measure for Decision Trees: Theoretical Foundations and Empirical Evaluation

• URL: http://arxiv.org/abs/2603.13241v1
• Date: Thu, 19 Feb 2026 19:17:50 GMT
• Title: A Hybrid Tsallis-Polarization Impurity Measure for Decision Trees: Theoretical Foundations and Empirical Evaluation
• Abstract summary: Integrated Tsallis Combination (ITC) is a hybrid impurity measure for decision tree learning.<n>The core innovation lies in the complementarity between Tsallis entropy's information-theoretic foundations and the polarization component's sensitivity to distributional asymmetry.<n>We show that simple parametric measures (Tsallis $=0.5$) achieve the highest average accuracy ($91.17%$), while ITC variants yield competitive results ($88.38-89.16%$) with strong theoretical guarantees.
• Score: 1.547549252134621
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We introduce the Integrated Tsallis Combination (ITC), a hybrid impurity measure for decision tree learning that combines normalized Tsallis entropy with an exponential polarization component. While many existing measures sacrifice theoretical soundness for computational efficiency or vice versa, ITC provides a mathematically principled framework that balances both aspects. The core innovation lies in the complementarity between Tsallis entropy's information-theoretic foundations and the polarization component's sensitivity to distributional asymmetry. We establish key theoretical properties-concavity under explicit parameter conditions, proper boundary conditions, and connections to classical measures-and provide a rigorous justification for the hybridization strategy. Through an extensive comparative evaluation on seven benchmark datasets comparing 23 impurity measures with five-fold repetition, we show that simple parametric measures (Tsallis $α=0.5$) achieve the highest average accuracy ($91.17\%$), while ITC variants yield competitive results ($88.38-89.16\%$) with strong theoretical guarantees. Statistical analysis (Friedman test: $χ^2=3.89$, $p=0.692$) reveals no significant global differences among top performers, indicating practical equivalence for many applications. ITC's value resides in its solid theoretical grounding-proven concavity under suitable conditions, flexible parameterization ($α$, $β$, $γ$), and computational efficiency $O(K)$-making it a rigorous, generalizable alternative when theoretical guarantees are paramount. We provide guidelines for measure selection based on application priorities and release an open-source implementation to foster reproducibility and further research.

Related papers

• Sharper Analysis of Single-Loop Methods for Bilevel Optimization [81.36947969465848]
  We establish sharper convergence results for single-loop approximate implicit differentiation (AID) and iterative differentiation (ITD) methods.<n>We prove that the error is $mathcalO(2)$, exactly matching the known lower bound and improving upon the previous $mathcalO(3)$ guarantee.
  arXiv  Detail & Related papers  (2026-07-11T11:41:42Z)
• Provably Adaptive Linear Approximation for the Shapley Value and Beyond [73.0940890296463]
  A fundamental and long-standing challenge is their efficient approximation.<n>We develop a linear-space algorithm that requires $P(|hatboldsymbol-boldsymbol|_2geq)leq $ for all commonly used semi-values.<n>Our algorithm allows explicit minimization of the mean square error for each specific utility function.
  arXiv  Detail & Related papers  (2026-04-09T16:38:14Z)
• Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guarantees [39.46018322554442]
  We establish a rigorous theoretical framework for low-rank knowledge distillation in language models.<n>We derive generalization bounds that characterize the fundamental trade-off between model compression and generalization capability.<n>Our results offer principled guidelines for rank selection, mathematically suggesting an optimal rank $r* = O(sqrtn)$ where $n$ is the sample size.
  arXiv  Detail & Related papers  (2026-03-22T14:10:27Z)
• On the Statistical Optimality of Optimal Decision Trees [6.920276126310231]
  We develop a comprehensive statistical theory for globally optimal empirical risk minimization trees under random design.<n>We first establish sharp oracle inequalities that bound the excess risk of the ERM estimator relative to the best possible approximation achievable by any tree with at most $L$ leaves.<n>We derive minimax optimal rates over a novel function class: the piecewise sparse heterogeneous anisotropic Besov (PSHAB) space.
  arXiv  Detail & Related papers  (2026-03-05T16:16:25Z)
• A Polynomial-Time Axiomatic Alternative to SHAP for Feature Attribution [0.0]
  We study feature attribution through the lens of cooperative game theory by formulating a class of XAI-TU games.<n>We propose a low-cost attribution rule, ESENSC_rev2, constructed by combining two-time closed-form rules.<n>Experiments show that ESENSC_rev2 closely approximates exact SHAP while substantially improving scalability.
  arXiv  Detail & Related papers  (2026-02-28T06:30:35Z)
• Principled Algorithms for Optimizing Generalized Metrics in Binary Classification [53.604375124674796]
  We introduce principled algorithms for optimizing generalized metrics, supported by $H$-consistency and finite-sample generalization bounds.<n>Our approach reformulates metric optimization as a generalized cost-sensitive learning problem.<n>We develop new algorithms, METRO, with strong theoretical performance guarantees.
  arXiv  Detail & Related papers  (2025-12-29T01:33:42Z)
• Likelihood-Preserving Embeddings for Statistical Inference [0.0]
  Modern machine learning embeddings provide powerful compression of high-dimensional data.<n>This paper develops a theory of likelihood-preserving embeddings.<n>Experiments on Gaussian and Cauchy distributions validate the sharp phase transition predicted by exponential family theory.
  arXiv  Detail & Related papers  (2025-12-27T16:21:55Z)
• Kernel-Based Nonparametric Tests For Shape Constraints [0.0]
  We derive statistical properties of the sample estimator and provide rigorous theoretical guarantees.<n>We introduce a joint Wald-type statistic to test for shape constraints over finite grids.
  arXiv  Detail & Related papers  (2025-10-19T08:07:33Z)
• A Fresh Look at Generalized Category Discovery through Non-negative Matrix Factorization [83.12938977698988]
  Generalized Category Discovery (GCD) aims to classify both base and novel images using labeled base data. Current approaches inadequately address the intrinsic optimization of the co-occurrence matrix $barA$ based on cosine similarity. We propose a Non-Negative Generalized Category Discovery (NN-GCD) framework to address these deficiencies.
  arXiv  Detail & Related papers  (2024-10-29T07:24:11Z)
• Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization [55.98760097296213]
  We introduce a new framework for online non-parametric LRE (OLRE) for the setting where pairs of iid observations $(x_t sim p, x'_t sim q)$ are observed over time. We provide theoretical guarantees for the performance of the OLRE method along with empirical validation in synthetic experiments.
  arXiv  Detail & Related papers  (2023-11-03T13:20:11Z)
• Pseudo-Spherical Contrastive Divergence [119.28384561517292]
  We propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum learning likelihood of energy-based models. PS-CD avoids the intractable partition function and provides a generalized family of learning objectives.
  arXiv  Detail & Related papers  (2021-11-01T09:17:15Z)
• Permutation Compressors for Provably Faster Distributed Nonconvex Optimization [68.8204255655161]
  We show that the MARINA method of Gorbunov et al (2021) can be considered as a state-of-the-art method in terms of theoretical communication complexity. Theory of MARINA to support the theory of potentially em correlated compressors, extends to the method beyond the classical independent compressors setting.
  arXiv  Detail & Related papers  (2021-10-07T09:38:15Z)
• A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers [3.167685495996986]
  This paper establishes a precise high-dimensional theory for boosting on separable data. Under a class of statistical models, we provide an exact analysis of the universality error of boosting. We also explicitly pin down the relation between the boosting test error and the optimal Bayes error.
  arXiv  Detail & Related papers  (2020-02-05T00:24:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.