Evaluation of Multi- and Single-objective Learning Algorithms for Imbalanced Data

• URL: http://arxiv.org/abs/2511.12191v1
• Date: Sat, 15 Nov 2025 12:54:17 GMT
• Title: Evaluation of Multi- and Single-objective Learning Algorithms for Imbalanced Data
• Authors: Szymon Wojciechowski, Michał Woźniak,
• Abstract summary: Machine learning tasks aim to find models that work well not for a single, but for a group of criteria, often opposing ones.<n>One solution is to propose an aggregate learning criterion and reduce the multi-objective learning task to a single-criteria optimization problem.<n>This article proposes a new, reliable way of evaluating algorithms based on multi-objective algorithms with methods that return single solutions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Many machine learning tasks aim to find models that work well not for a single, but for a group of criteria, often opposing ones. One such example is imbalanced data classification, where, on the one hand, we want to achieve the best possible classification quality for data from the minority class without degrading the classification quality of the majority class. One solution is to propose an aggregate learning criterion and reduce the multi-objective learning task to a single-criteria optimization problem. Unfortunately, such an approach is characterized by ambiguity of interpretation since the value of the aggregated criterion does not indicate the value of the component criteria. Hence, there are more and more proposals for algorithms based on multi-objective optimization (MOO), which can simultaneously optimize multiple criteria. However, such an approach results in a set of multiple non-dominated solutions (Pareto front). The selection of a single solution from the Pareto front is a challenge itself, and much attention is paid to the issue of how to select it considering user preferences, as well as how to compare solutions returned by different MOO algorithms among themselves. Thus, a significant gap has been identified in the classifier evaluation methodology, i.e., how to reliably compare methods returning single solutions with algorithms returning solutions in the form of Pareto fronts. To fill the aforementioned gap, this article proposes a new, reliable way of evaluating algorithms based on multi-objective algorithms with methods that return single solutions while pointing out solutions from a Pareto front tailored to the user's preferences. This work focuses only on algorithm comparison, not their learning. The algorithms selected for this study are illustrative to help understand the proposed approach.

Related papers

• Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gains [67.71020482405343]
  We study how to design algorithms that can leverage additional information in the form of rating gap.<n>We present new algorithms that can achieve faster statistical rates than DPO in presence of accurate rating gap information.
  arXiv  Detail & Related papers  (2026-01-31T08:38:21Z)
• A Review on Single-Problem Multi-Attempt Heuristic Optimization [6.778082328635129]
  In certain real-world optimization scenarios, practitioners are not interested in solving multiple problems but rather in finding the best solution to a single, specific problem.<n>When the computational budget is large relative to the cost of evaluating a candidate solution, multiple alternatives can be tried to solve the same given problem.<n>The sequential selection of which alternative to try next is crucial for efficiently identifying the one that provides the best possible solution.
  arXiv  Detail & Related papers  (2025-09-30T14:28:28Z)
• It's My Data Too: Private ML for Datasets with Multi-User Training Examples [9.18252846535411]
  We first provide a carefully chosen definition of user-level DP under the multi-attribution model.<n>We propose a greedy baseline algorithm for the contribution bounding problem.<n>We study variants of this baseline algorithm that optimize the subset chosen using different techniques and criteria.
  arXiv  Detail & Related papers  (2025-03-05T16:02:09Z)
• Towards Fast Algorithms for the Preference Consistency Problem Based on Hierarchical Models [4.007697401483925]
  We construct and compare algorithmic approaches to solve the Consistency Problem for preference statements based on hierarchical models. An instance is consistent if there exists an hierarchical model on the evaluation functions that induces an order relation on the alternatives. We develop three approaches to solve this decision problem.
  arXiv  Detail & Related papers  (2024-10-31T13:48:46Z)
• An incremental preference elicitation-based approach to learning potentially non-monotonic preferences in multi-criteria sorting [53.36437745983783]
  We first construct a max-margin optimization-based model to model potentially non-monotonic preferences. We devise information amount measurement methods and question selection strategies to pinpoint the most informative alternative in each iteration. Two incremental preference elicitation-based algorithms are developed to learn potentially non-monotonic preferences.
  arXiv  Detail & Related papers  (2024-09-04T14:36:20Z)
• Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization [52.80408805368928]
  We introduce a novel greedy-style subset selection algorithm for batch acquisition. Our experiments on the red fluorescent proteins show that our proposed method achieves the baseline performance in 1.69x fewer queries.
  arXiv  Detail & Related papers  (2024-06-21T05:57:08Z)
• Algorithm Selection on a Meta Level [58.720142291102135]
  We introduce the problem of meta algorithm selection, which essentially asks for the best way to combine a given set of algorithm selectors. We present a general methodological framework for meta algorithm selection as well as several concrete learning methods as instantiations of this framework.
  arXiv  Detail & Related papers  (2021-07-20T11:23:21Z)
• Learning to Select Base Classes for Few-shot Classification [96.92372639495551]
  We use the Similarity Ratio as an indicator for the generalization performance of a few-shot model. We then formulate the base class selection problem as a submodular optimization problem over Similarity Ratio.
  arXiv  Detail & Related papers  (2020-04-01T09:55:18Z)
• Extreme Algorithm Selection With Dyadic Feature Representation [78.13985819417974]
  We propose the setting of extreme algorithm selection (XAS) where we consider fixed sets of thousands of candidate algorithms. We assess the applicability of state-of-the-art AS techniques to the XAS setting and propose approaches leveraging a dyadic feature representation.
  arXiv  Detail & Related papers  (2020-01-29T09:40:58Z)
• Optimal Clustering from Noisy Binary Feedback [75.17453757892152]
  We study the problem of clustering a set of items from binary user feedback. We devise an algorithm with a minimal cluster recovery error rate. For adaptive selection, we develop an algorithm inspired by the derivation of the information-theoretical error lower bounds.
  arXiv  Detail & Related papers  (2019-10-14T09:18:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.