Related papers: When to Truncate a Feature Ranking: A Residual-Overlap Stopping Rule for Subset Selection

When to Truncate a Feature Ranking: A Residual-Overlap Stopping Rule for Subset Selection

URL: http://arxiv.org/abs/2606.31686v1
Date: Tue, 30 Jun 2026 14:00:29 GMT
Title: When to Truncate a Feature Ranking: A Residual-Overlap Stopping Rule for Subset Selection
Abstract summary: This paper develops a distributional framework for transforming supervised feature rankings into class-independent subsets.<n>An empirical comparison on high-dimensional genomic datasets illustrates that the rule can reduce tens of thousands of variables to a few dozen.
Score: 0.0
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Abstract: Feature rankings are widely used in supervised feature selection because they are simple, scalable and easy to interpret. Variables are first ranked by a relevance score, and a subset is then obtained by retaining the top-ranked variables. Although the first stage has been extensively studied, the second is often governed by an arbitrary cardinality, an empirical threshold or cross-validation, without a direct interpretation. This raises a basic question: given a feature ranking, when is there enough accumulated class-separation evidence to stop selecting features? This paper develops a distributional framework for transforming supervised feature rankings into class-independent subsets through an explicit risk-calibrated stopping rule. For each variable and each pair of classes, marginal separation is measured by the Bhattacharyya coefficient between the corresponding class-conditional distributions. The proposed method selects a single global subset shared by all classes by retaining the shortest prefix of a ranking whose residual product overlap falls below a prescribed threshold for every relevant class contrast. We derive binary and multiclass Bayes-risk bounds for the labelled product marginal problem, and obtain prior-dependent and prior-free calibrations of the residual-overlap threshold from a target all-pairs risk level. An empirical comparison on high-dimensional genomic datasets illustrates that the rule can reduce tens of thousands of variables to a few dozen while maintaining predictive performance statistically comparable to the all-features baseline. As the stopping rule only requires one-dimensional marginal overlap estimates and scans a precomputed ranking, it is well suited to very high-dimensional settings where exhaustive subset search is infeasible and interpretable truncation of feature rankings is essential.

Related papers

How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? [58.740078141879984]
Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on.<n>Recent work in causality-based domain generalization has shown how shared causal structure can induce invariant predictors.<n>In this paper, we study when full or partial causal knowledge can provably improve supervised domain adaptation.
arXiv Detail & Related papers (2026-06-10T21:07:49Z)
Fair Decisions from Calibrated Scores: Achieving Optimal Classification While Satisfying Sufficiency [2.0686600920324163]
Binary classification based on predicted probabilities (scores) is a fundamental task in supervised machine learning.<n>We present an exact solution for optimal binary classification under sufficiency, assuming finite sets of group-calibrated scores.
arXiv Detail & Related papers (2026-02-07T00:26:40Z)
The Multiclass Score-Oriented Loss (MultiSOL) on the Simplex [4.014524824655106]
In supervised binary classification, score-oriented losses have been introduced with the aim of optimizing a chosen performance metric directly during the training phase.<n>In this paper, we use a recently introduced multidimensional threshold-based classification framework to extend such score-oriented losses to multiclass classification.<n>As also demonstrated by several classification experiments, this proposed family of losses is designed to preserve the main advantages observed in the binary setting.
arXiv Detail & Related papers (2025-11-27T16:20:55Z)
Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distribution to Multivariate Settings [0.14504054468850666]
Conformal prediction (CP) constructs uncertainty sets for model outputs with finite-sample coverage guarantees.<n>We show that optimal assignment is piecewise-constant across a fixed polyhedral partition of the score space.<n>This allows us to characterize the entire prediction set tractably, and provides the machinery to address a deeper limitation of prediction sets.
arXiv Detail & Related papers (2025-11-19T05:59:01Z)
A comparative analysis of rank aggregation methods for the partial label ranking problem [13.081896295252832]
The label ranking problem is a supervised learning scenario in which the learner predicts a total order of the class labels for a given input instance.<n>This paper explores several alternative aggregation methods for this critical step, including scoring-based and non-parametric probabilistic-based rank aggregation approaches.
arXiv Detail & Related papers (2025-02-24T11:44:43Z)
Probably Approximately Precision and Recall Learning [60.00180898830079]
A key challenge in machine learning is the prevalence of one-sided feedback.<n>We introduce a Probably Approximately Correct (PAC) framework in which hypotheses are set functions that map each input to a set of labels.<n>We develop new algorithms that learn from positive data alone, achieving optimal sample complexity in the realizable case.
arXiv Detail & Related papers (2024-11-20T04:21:07Z)
Generalization bounds for regression and classification on adaptive covering input domains [1.4141453107129398]
We focus on the generalization bound, which serves as an upper limit for the generalization error. In the case of classification tasks, we treat the target function as a one-hot, a piece-wise constant function, and employ 0/1 loss for error measurement.
arXiv Detail & Related papers (2024-07-29T05:40:08Z)
Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
arXiv Detail & Related papers (2023-11-03T08:39:35Z)
Deep Imbalanced Regression via Hierarchical Classification Adjustment [50.19438850112964]
Regression tasks in computer vision are often formulated into classification by quantizing the target space into classes. The majority of training samples lie in a head range of target values, while a minority of samples span a usually larger tail range. We propose to construct hierarchical classifiers for solving imbalanced regression tasks. Our novel hierarchical classification adjustment (HCA) for imbalanced regression shows superior results on three diverse tasks.
arXiv Detail & Related papers (2023-10-26T04:54:39Z)
Mitigating Word Bias in Zero-shot Prompt-based Classifiers [55.60306377044225]
We show that matching class priors correlates strongly with the oracle upper bound performance. We also demonstrate large consistent performance gains for prompt settings over a range of NLP tasks.
arXiv Detail & Related papers (2023-09-10T10:57:41Z)
Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification [94.55805516167369]
We propose a new approach for binary classification from m U-sets for $mge2$. Our key idea is to consider an auxiliary classification task called surrogate set classification (SSC)
arXiv Detail & Related papers (2021-02-01T07:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.