Feature Selection Based on Wasserstein Distance

• URL: http://arxiv.org/abs/2411.07217v3
• Date: Wed, 13 Nov 2024 23:09:38 GMT
• Title: Feature Selection Based on Wasserstein Distance
• Abstract summary: This paper presents a novel feature selection method leveraging the Wasserstein distance to improve feature selection in machine learning. Our approach uses the Wasserstein distance to assess feature similarity, inherently capturing class relationships and making it robust to noisy labels. Our analysis shows that the Wasserstein distance-based feature selection method effectively reduces the impact of noisy labels without relying on specific noise models.
• Score: 1.450405446885067
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents a novel feature selection method leveraging the Wasserstein distance to improve feature selection in machine learning. Unlike traditional methods based on correlation or Kullback-Leibler (KL) divergence, our approach uses the Wasserstein distance to assess feature similarity, inherently capturing class relationships and making it robust to noisy labels. We introduce a Markov blanket-based feature selection algorithm and demonstrate its effectiveness. Our analysis shows that the Wasserstein distance-based feature selection method effectively reduces the impact of noisy labels without relying on specific noise models. We provide a lower bound on its effectiveness, which remains meaningful even in the presence of noise. Experimental results across multiple datasets demonstrate that our approach consistently outperforms traditional methods, particularly in noisy settings.

Related papers

• Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio [58.612001688217056]
  Inference-time watermarks for autoregressive models are unfit for continuous modalities due to discretization inconsistencies.<n>We propose an elegant solution for powerful and robust watermarking of synthetic audio.
  arXiv  Detail & Related papers  (2026-05-25T15:43:20Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Robust variable selection for spatial point processes observed with noise [1.8047694351309207]
  We propose a method variable selection in the intensity function of point processes that combines sparsity-promoting estimation with noise-robust model selection.<n>We demonstrate that an approach reliably recovers true spatial cos under diverse noise scenarios and improves both selection stability.<n>This shows the practical utility of the method, which provides a systematic framework for robust variable selection in point-process models under noise.
  arXiv  Detail & Related papers  (2025-10-29T14:18:08Z)
• Benchmarking noisy label detection methods [0.3154269505086154]
  Label noise is a common problem in real-world datasets, affecting both model training and validation.<n>We perform a comprehensive benchmark of detection methods by decomposing them into three fundamental components.<n>We identify that in-sample information gathering using average probability aggregation combined with the logit margin achieves the best results.
  arXiv  Detail & Related papers  (2025-10-17T20:55:26Z)
• Extracting Clean and Balanced Subset for Noisy Long-tailed Classification [66.47809135771698]
  We develop a novel pseudo labeling method using class prototypes from the perspective of distribution matching. By setting a manually-specific probability measure, we can reduce the side-effects of noisy and long-tailed data simultaneously. Our method can extract this class-balanced subset with clean labels, which brings effective performance gains for long-tailed classification with label noise.
  arXiv  Detail & Related papers  (2024-04-10T07:34:37Z)
• Skeleton-Based Human Action Recognition with Noisy Labels [43.58994208412764]
  Label noise negatively affects the model's training, resulting in lower recognition quality. NoiseEraSAR integrates global sample selection, co-teaching, and Cross-Modal Mixture-of-Experts strategies. Our proposed approach demonstrates better performance on the established benchmark, setting new state-of-the-art standards.
  arXiv  Detail & Related papers  (2024-03-15T02:42:28Z)
• Towards stable real-world equation discovery with assessing differentiating quality influence [52.2980614912553]
  We propose alternatives to the commonly used finite differences-based method. We evaluate these methods in terms of applicability to problems, similar to the real ones, and their ability to ensure the convergence of equation discovery algorithms.
  arXiv  Detail & Related papers  (2023-11-09T23:32:06Z)
• Bandit-Driven Batch Selection for Robust Learning under Label Noise [20.202806541218944]
  We introduce a novel approach for batch selection in Gradient Descent (SGD) training, leveraging bandit algorithms. Our methodology focuses on optimizing the learning process in the presence of label noise, a prevalent issue in real-world datasets.
  arXiv  Detail & Related papers  (2023-10-31T19:19:01Z)
• Combating Label Noise With A General Surrogate Model For Sample Selection [84.61367781175984]
  We propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically. We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets.
  arXiv  Detail & Related papers  (2023-10-16T14:43:27Z)
• Score-based Source Separation with Applications to Digital Communication Signals [72.6570125649502]
  We propose a new method for separating superimposed sources using diffusion-based generative models. Motivated by applications in radio-frequency (RF) systems, we are interested in sources with underlying discrete nature. Our method can be viewed as a multi-source extension to the recently proposed score distillation sampling scheme.
  arXiv  Detail & Related papers  (2023-06-26T04:12:40Z)
• Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile [78.1212767880785]
  meta-learner is prone to overfitting since there are only a few available samples. When handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise. We present Eigen-Reptile (ER) that updates the meta- parameters with the main direction of historical task-specific parameters.
  arXiv  Detail & Related papers  (2022-06-04T08:48:02Z)
• UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning [89.56465237941013]
  We propose UNICON, a simple yet effective sample selection method which is robust to high label noise. We obtain an 11.4% improvement over the current state-of-the-art on CIFAR100 dataset with a 90% noise rate.
  arXiv  Detail & Related papers  (2022-03-28T07:36:36Z)
• Optimizing Speech Emotion Recognition using Manta-Ray Based Feature Selection [1.4502611532302039]
  We show that concatenation of features, extracted by using different existing feature extraction methods can boost the classification accuracy. We also perform a novel application of Manta Ray optimization in speech emotion recognition tasks that resulted in a state-of-the-art result.
  arXiv  Detail & Related papers  (2020-09-18T16:09:34Z)
• Differentiable Unsupervised Feature Selection based on a Gated Laplacian [7.970954821067042]
  We propose a differentiable loss function that combines the Laplacian score, which favors low-frequency features, with a gating mechanism for feature selection. We mathematically motivate the proposed approach and demonstrate that in the high noise regime, it is crucial to compute the Laplacian on the gated inputs, rather than on the full feature set.
  arXiv  Detail & Related papers  (2020-07-09T11:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.