Regionalized Metric Framework: A Novel Approach for Evaluating Multimodal Multi-Objective Optimization Algorithms

• URL: http://arxiv.org/abs/2506.00468v1
• Date: Sat, 31 May 2025 08:36:55 GMT
• Title: Regionalized Metric Framework: A Novel Approach for Evaluating Multimodal Multi-Objective Optimization Algorithms
• Authors: Jintai Chen, Fangqing Liu, Xueming Yan, Han Huang,
• Abstract summary: This study proposes an evaluation metric based on a Regionalized Metric Framework.<n>The algorithm divides the set of solutions to be evaluated into three regions, and evaluates each solution according to a unique scoring function for each region.
• Score: 11.848588480889607
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study aims to optimize the evaluation metric of multimodal multi-objective optimization problems using a Regionalized Metric Framework, which provides a certain boost to research in this field. Existing evaluation metrics usually use the reference set as the evaluation basis, which inevitably leads to reference set dependence. To optimize this problem, this study proposes an evaluation metric based on a Regionalized Metric Framework. The algorithm divides the set of solutions to be evaluated into three regions, and evaluates each solution according to a unique scoring function for each region, which is combined to form the evaluation value of the solution set. To verify the feasibility of this method, a comparative experiment was conducted in this study. The results of the experiment are roughly the same as the trend of existing indicators, and at the same time, it can accurately judge the advantages and disadvantages of points equidistant from the reference set. Our method provides a new perspective for further research on evaluation metrics for multimodal multi-objective optimization algorithms.

Related papers

• MO-IOHinspector: Anytime Benchmarking of Multi-Objective Algorithms using IOHprofiler [0.7418044931036347]
  We propose a new software tool which uses principles from unbounded archiving as a logging structure.<n>This leads to a clearer separation between experimental design and subsequent analysis decisions.
  arXiv  Detail & Related papers  (2024-12-10T12:00:53Z)
• A Novel Pareto-optimal Ranking Method for Comparing Multi-objective Optimization Algorithms [2.889178722750616]
  This paper proposes a novel multi-metric comparison method to rank the performance of multi-/many-objective optimization algorithms.<n>Four different techniques are proposed to rank algorithms based on their contribution at each Pareto level.<n>The techniques have broad applications in science and engineering, particularly in areas where multiple metrics are used for comparisons.
  arXiv  Detail & Related papers  (2024-11-27T02:34:54Z)
• Absolute Ranking: An Essential Normalization for Benchmarking Optimization Algorithms [0.0]
  evaluating performance across optimization algorithms on many problems presents a complex challenge due to the diversity of numerical scales involved. This paper extensively explores the problem, making a compelling case to underscore the issue and conducting a thorough analysis of its root causes. Building on this research, this paper introduces a new mathematical model called "absolute ranking" and a sampling-based computational method.
  arXiv  Detail & Related papers  (2024-09-06T00:55:03Z)
• FSDEM: Feature Selection Dynamic Evaluation Metric [1.54369283425087]
  The proposed metric is a dynamic metric with two properties that can be used to evaluate both the performance and the stability of a feature selection algorithm.<n>We conduct several empirical experiments to illustrate the use of the proposed metric in the successful evaluation of feature selection algorithms.
  arXiv  Detail & Related papers  (2024-08-26T12:49:41Z)
• Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition [70.60872754129832]
  First NeurIPS competition on unlearning sought to stimulate the development of novel algorithms. Nearly 1,200 teams from across the world participated. We analyze top solutions and delve into discussions on benchmarking unlearning.
  arXiv  Detail & Related papers  (2024-06-13T12:58:00Z)
• Optimal Baseline Corrections for Off-Policy Contextual Bandits [61.740094604552475]
  We aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. We propose a single framework built on their equivalence in learning scenarios. Our framework enables us to characterize the variance-optimal unbiased estimator and provide a closed-form solution for it.
  arXiv  Detail & Related papers  (2024-05-09T12:52:22Z)
• A new fuzzy multi-attribute group decision-making method based on TOPSIS and optimization models [3.697049647195136]
  A new method is proposed for multi-attribute group decision-making in interval-valued intuitionistic fuzzy sets. By minimizing the sum of differences between individual evaluations and the overallconsistent evaluations of all experts, a new optimization model is established for determining expert weights. The complete fuzzy multi-attribute group decision-making algorithm is formulated, which can give full play to the advantages of subjective and objective weighting methods.
  arXiv  Detail & Related papers  (2023-11-27T15:41:30Z)
• Best-Effort Adaptation [62.00856290846247]
  We present a new theoretical analysis of sample reweighting methods, including bounds holding uniformly over the weights. We show how these bounds can guide the design of learning algorithms that we discuss in detail. We report the results of a series of experiments demonstrating the effectiveness of our best-effort adaptation and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-05-10T00:09:07Z)
• Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
  arXiv  Detail & Related papers  (2023-03-21T14:24:58Z)
• Towards Better Understanding Attribution Methods [77.1487219861185]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
  arXiv  Detail & Related papers  (2022-05-20T20:50:17Z)
• A Statistical Analysis of Summarization Evaluation Metrics using Resampling Methods [60.04142561088524]
  We find that the confidence intervals are rather wide, demonstrating high uncertainty in how reliable automatic metrics truly are. Although many metrics fail to show statistical improvements over ROUGE, two recent works, QAEval and BERTScore, do in some evaluation settings.
  arXiv  Detail & Related papers  (2021-03-31T18:28:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.