Objective-Induced Bias and Search Dynamics in Multiobjective Unsupervised Feature Selection

• URL: http://arxiv.org/abs/2605.21561v1
• Date: Wed, 20 May 2026 15:14:00 GMT
• Title: Objective-Induced Bias and Search Dynamics in Multiobjective Unsupervised Feature Selection
• Abstract summary: We study the choice of evaluation objective, the direction of subset-size regularisation, and the initialisation strategy.<n>We show that objective design is central to effective multiobjective unsupervised feature selection.
• Score: 2.2282508171588806
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised feature selection is commonly formulated as a multiobjective optimisation problem that jointly optimises subset quality and subset size. Yet the behaviour of this formulation depends critically on the choice of evaluation objective, the direction of subset-size regularisation, and the initialisation strategy. We study these factors in a controlled setting using a synthetic dataset with known informative, redundant, and irrelevant feature types. Six formulations are compared by combining three evaluation objectives: accuracy, silhouette score, and PCA reconstruction loss with subset-size minimisation or maximisation. The results show that formulation strongly affects both search dynamics and the quality of the resulting Pareto front. Silhouette-based formulations exhibit a strong bias toward trivial low-cardinality solutions and remain weak proxies for predictive performance. In contrast, the proposed PCA loss objective produces compact subsets with test accuracy comparable to subsets obtained by directly optimising supervised accuracy. Overall, the study shows that objective design is central to effective multiobjective unsupervised feature selection.

Related papers

• Preference Data Selection for Mitigating the Alignment Tax in Large Language Models [52.18310687523229]
  BALIGN is a balanced data selection strategy that mitigates catastrophic forgetting while optimizing alignment efficacy.<n>By aggregating three key data-centric features into a unified composite risk score, BALIGN systematically filters out high-risk preference samples.<n>Experiments on standard human preference datasets demonstrate that BALIGN strongly preserves foundational capabilities without compromising alignment gains.
  arXiv  Detail & Related papers  (2026-08-25T07:58:42Z)
• Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs [55.715754117956514]
  Multiple Sequence Alignments provide protein language models with explicit evolutionary context.<n>MSA subsampling is unavoidable under limited token budgets.<n>We introduce AP-REASONER, which infers a fixed-budget MSA subset using factor-graph reasoning.
  arXiv  Detail & Related papers  (2026-07-24T13:55:21Z)
• Stochastic set-valued optimization and its application to robust learning [0.0]
  We develop a set-valued optimization (SVO) framework tailored for robust machine learning.<n>In the SVO setting, each decision variable is mapped to a set of objective values, and optimality is defined via set relations.
  arXiv  Detail & Related papers  (2026-03-18T13:07:53Z)
• Structural bias in multi-objective optimisation [5.869782631289138]
  Structural bias (SB) refers to systematic preferences of an optimisation algorithm for particular regions of the search space.<n> SB is studied extensively in single-objective optimisation, but its role in multi-objective optimisation remains largely unexplored.<n>In this paper, we extend the concept of structural bias to the multi-objective setting and propose a methodology to study it in isolation from fitness-driven guidance.
  arXiv  Detail & Related papers  (2026-02-06T14:43:28Z)
• Enhanced Ideal Objective Vector Estimation for Evolutionary Multi-Objective Optimization [24.71649541757315]
  The ideal objective vector, which comprises the optimal values of the $m$ objective functions in an $m$-objective optimization problem, is an important concept in evolutionary multi-objective optimization.<n>This paper reveals that the population-based estimation method can only work on simple problems but falls short on problems with substantial bias.<n>A plug-and-play component called enhanced ideal objective vector estimation (EIE) is introduced for multi-objective evolutionary algorithms.
  arXiv  Detail & Related papers  (2025-05-28T02:36:45Z)
• Pareto Optimization with Robust Evaluation for Noisy Subset Selection [34.83487850400559]
  Subset selection is a fundamental problem in optimization, which has a wide range of applications such as influence and sparse regression.<n>Previous algorithms, including the greedy algorithm and evolutionary evolutionary POSS, either struggle in noisy environments or consume excessive computational resources.<n>We propose a novel approach based on Pareto Optimization with Robust Evaluation for noisy subset selection (PORE), which maximizes a robust evaluation function and minimizes the subset size simultaneously.
  arXiv  Detail & Related papers  (2025-01-12T14:04:20Z)
• Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
  We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences.<n>To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model.<n>Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
  arXiv  Detail & Related papers  (2024-05-26T05:38:50Z)
• Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [103.12563033438715]
  Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values. Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives. We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
  arXiv  Detail & Related papers  (2024-02-29T12:12:30Z)
• Enhancing Neural Subset Selection: Integrating Background Information into Set Representations [53.15923939406772]
  We show that when the target value is conditioned on both the input set and subset, it is essential to incorporate an textitinvariant sufficient statistic of the superset into the subset of interest. This ensures that the output value remains invariant to permutations of the subset and its corresponding superset, enabling identification of the specific superset from which the subset originated.
  arXiv  Detail & Related papers  (2024-02-05T16:09:35Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Alternating Objectives Generates Stronger PGD-Based Adversarial Attacks [78.2700757742992]
  Projected Gradient Descent (PGD) is one of the most effective and conceptually simple algorithms to generate such adversaries. We experimentally verify this assertion on a synthetic-data example and by evaluating our proposed method across 25 different $ell_infty$-robust models and 3 datasets. Our strongest adversarial attack outperforms all of the white-box components of AutoAttack ensemble.
  arXiv  Detail & Related papers  (2022-12-15T17:44:31Z)
• Bi-objective Ranking and Selection Using Stochastic Kriging [0.0]
  We consider bi-objective ranking and selection problems in which the two objective outcomes have been observed with uncertainty. We propose a novel Bayesian bi-objective ranking and selection method that sequentially allocates extra samples to competitive solutions. Experimental results show that the proposed method outperforms the standard allocation method, as well as a well-known state-of-the-art algorithm.
  arXiv  Detail & Related papers  (2022-09-05T23:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.