Strategic Feature Selection

• URL: http://arxiv.org/abs/2606.18867v1
• Date: Wed, 17 Jun 2026 09:47:15 GMT
• Title: Strategic Feature Selection
• Abstract summary: We study strategic classification through feature selection and its interaction with ridge regularization.<n>We provide a fine-grained characterization of the performance of a feature subset under optimal regularization.<n>Motivated by this characterization, we develop a practical algorithm for jointly choosing the feature set and the level of ridge regularization.
• Score: 62.74494413516484
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When algorithmic predictors inform resource allocation in high-stakes domains such as healthcare, these predictors must account for strategic manipulation of input features. The typical solution is to redesign the predictor itself to explicitly account for strategic interactions. In practice, however, decision makers are often constrained to adjusting coarser levers within existing prediction pipelines. For example, healthcare organizations often select which features to exclude based on perceived manipulability, while using standard regularization procedures to shrink the coefficients of retained features. In this work, we initiate a formal study of strategic classification through feature selection and its interaction with ridge regularization. Our main finding is that excluding individual features based on their manipulability alone is generally suboptimal. We provide a fine-grained characterization of the performance of a feature subset under optimal regularization, yielding new insights for policy design. Motivated by this characterization, we develop a practical algorithm for jointly choosing the feature set and the level of ridge regularization. Through a real-world case study on a healthcare payments benchmark, we illustrate how our algorithm can guide the design of coarse policy levers in practice. Our results provide a principled, practical framework for mitigating the effects of strategic behavior in algorithmic decision-making systems.

Related papers

• Linear Strategic Classification with Endogenous Improvements [6.798190446932253]
  We study an improvement-aware variant in which strategic responses can induce genuine changes in outcome-relevant features.<n>Agents choose post-deployment feature vectors strategically, and labels are then generated according to a stable conditional outcome law.
  arXiv  Detail & Related papers  (2026-05-31T12:22:30Z)
• Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance [86.46794021499511]
  We show a previously underexplored gap between strategy usage and strategy executability.<n>We propose Selective Strategy Retrieval (SSR), a test-time framework that explicitly models executability.<n> SSR yields reliable and consistent improvements over direct solving, in-context learning, and single-source guidance.
  arXiv  Detail & Related papers  (2026-02-26T03:34:23Z)
• A Multi-Component Reward Function with Policy Gradient for Automated Feature Selection with Dynamic Regularization and Bias Mitigation [0.0]
  Static feature exclusion strategies fail to prevent bias when hidden dependencies influence the model predictions.<n>We develop a reinforcement learning framework that integrates bias mitigation and automated feature selection within a single learning process.<n>We aim to provide a flexible and generalizable way to select features in environments where predictors are correlated and biases can inadvertently re-emerge.
  arXiv  Detail & Related papers  (2025-10-09T22:45:38Z)
• In-Context Learning Strategies Emerge Rationally [36.21854069812912]
  We show that when trained to learn a mixture of tasks, strategies learned by a model for performing ICL can be captured by a family of Bayesian predictors.<n>We develop a hierarchical Bayesian framework that almost perfectly predicts Transformer next-token predictions throughout training.<n>Our work advances an explanatory and predictive account of ICL grounded in tradeoffs between strategy loss and complexity.
  arXiv  Detail & Related papers  (2025-06-21T23:49:08Z)
• Non-linear Welfare-Aware Strategic Learning [10.448052192725168]
  This paper studies algorithmic decision-making in the presence of strategic individual behaviors.<n>We first generalize the agent best response model in previous works to the non-linear setting.<n>We show the three welfare can attain the optimum simultaneously only under restrictive conditions.
  arXiv  Detail & Related papers  (2024-05-03T01:50:03Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Randomized Adversarial Style Perturbations for Domain Generalization [49.888364462991234]
  We propose a novel domain generalization technique, referred to as Randomized Adversarial Style Perturbation (RASP) The proposed algorithm perturbs the style of a feature in an adversarial direction towards a randomly selected class, and makes the model learn against being misled by the unexpected styles observed in unseen target domains. We evaluate the proposed algorithm via extensive experiments on various benchmarks and show that our approach improves domain generalization performance, especially in large-scale benchmarks.
  arXiv  Detail & Related papers  (2023-04-04T17:07:06Z)
• Off-Policy Evaluation with Policy-Dependent Optimization Response [90.28758112893054]
  We develop a new framework for off-policy evaluation with a textitpolicy-dependent linear optimization response. We construct unbiased estimators for the policy-dependent estimand by a perturbation method. We provide a general algorithm for optimizing causal interventions.
  arXiv  Detail & Related papers  (2022-02-25T20:25:37Z)
• Stochastic batch size for adaptive regularization in deep network optimization [63.68104397173262]
  We propose a first-order optimization algorithm incorporating adaptive regularization applicable to machine learning problems in deep learning framework. We empirically demonstrate the effectiveness of our algorithm using an image classification task based on conventional network models applied to commonly used benchmark datasets.
  arXiv  Detail & Related papers  (2020-04-14T07:54:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.