Data Valuation with Gradient Similarity

• URL: http://arxiv.org/abs/2405.08217v1
• Date: Mon, 13 May 2024 22:10:00 GMT
• Title: Data Valuation with Gradient Similarity
• Authors: Nathaniel J. Evans, Gordon B. Mills, Guanming Wu, Xubo Song, Shannon McWeeney,
• Abstract summary: Data Valuation algorithms quantify the value of each sample in a dataset based on its contribution or importance to a given predictive task. We present a simple alternative to existing methods, termed Data Valuation with Gradient Similarity (DVGS) Our approach has the ability to rapidly and accurately identify low-quality data, which can reduce the need for expert knowledge and manual intervention in data cleaning tasks.
• Score: 1.997283751398032
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-quality data is crucial for accurate machine learning and actionable analytics, however, mislabeled or noisy data is a common problem in many domains. Distinguishing low- from high-quality data can be challenging, often requiring expert knowledge and considerable manual intervention. Data Valuation algorithms are a class of methods that seek to quantify the value of each sample in a dataset based on its contribution or importance to a given predictive task. These data values have shown an impressive ability to identify mislabeled observations, and filtering low-value data can boost machine learning performance. In this work, we present a simple alternative to existing methods, termed Data Valuation with Gradient Similarity (DVGS). This approach can be easily applied to any gradient descent learning algorithm, scales well to large datasets, and performs comparably or better than baseline valuation methods for tasks such as corrupted label discovery and noise quantification. We evaluate the DVGS method on tabular, image and RNA expression datasets to show the effectiveness of the method across domains. Our approach has the ability to rapidly and accurately identify low-quality data, which can reduce the need for expert knowledge and manual intervention in data cleaning tasks.

Related papers

• Towards Explainable Automated Data Quality Enhancement without Domain Knowledge [0.0]
  We propose a comprehensive framework designed to automatically assess and rectify data quality issues in any given dataset. Our primary objective is to address three fundamental types of defects: absence, redundancy, and incoherence. We adopt a hybrid approach that integrates statistical methods with machine learning algorithms.
  arXiv  Detail & Related papers  (2024-09-16T10:08:05Z)
• Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyond [38.89457061559469]
  We propose an innovative methodology that automates dataset creation with negligible cost and high efficiency. We provide open-source software that incorporates existing methods for label error detection, robust learning under noisy and biased data. We design three benchmark datasets focused on label noise detection, label noise learning, and class-imbalanced learning.
  arXiv  Detail & Related papers  (2024-08-21T04:45:12Z)
• Neural Dynamic Data Valuation [4.286118155737111]
  We propose a novel data valuation method from the perspective of optimal control, named the neural dynamic data valuation (NDDV) Our method has solid theoretical interpretations to accurately identify the data valuation via the sensitivity of the data optimal control state. In addition, we implement a data re-weighting strategy to capture the unique features of data points, ensuring fairness through the interaction between data points and the mean-field states.
  arXiv  Detail & Related papers  (2024-04-30T13:39:26Z)
• Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution [62.71425232332837]
  We show that training amortized models with noisy labels is inexpensive and surprisingly effective. This approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
  arXiv  Detail & Related papers  (2024-01-29T03:42:37Z)
• OpenDataVal: a Unified Benchmark for Data Valuation [38.15852021170501]
  We introduce OpenDataVal, an easy-to-use and unified benchmark framework for data valuation. OpenDataVal provides an integrated environment that includes eleven different state-of-the-art data valuation algorithms. We perform benchmarking analysis using OpenDataVal, quantifying and comparing the efficacy of state-of-the-art data valuation approaches.
  arXiv  Detail & Related papers  (2023-06-18T14:38:29Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation [54.899751055620904]
  We study the phenomenon of memorization with turn-over dropout, an efficient method to estimate influence and memorization, for data with true labels (real data) and data with random labels (random data) Our main findings are: (i) For both real data and random data, the optimization of easy examples (e.g., real data) and difficult examples (e.g., random data) are conducted by the network simultaneously, with easy ones at a higher speed; (ii) For real data, a correct difficult example in the training dataset is more informative than an easy one.
  arXiv  Detail & Related papers  (2021-12-16T11:34:23Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Low-Regret Active learning [64.36270166907788]
  We develop an online learning algorithm for identifying unlabeled data points that are most informative for training. At the core of our work is an efficient algorithm for sleeping experts that is tailored to achieve low regret on predictable (easy) instances.
  arXiv  Detail & Related papers  (2021-04-06T22:53:45Z)
• Weak Adaptation Learning -- Addressing Cross-domain Data Insufficiency with Weak Annotator [2.8672054847109134]
  In some target problem domains, there are not many data samples available, which could hinder the learning process. We propose a weak adaptation learning (WAL) approach that leverages unlabeled data from a similar source domain. Our experiments demonstrate the effectiveness of our approach in learning an accurate classifier with limited labeled data in the target domain.
  arXiv  Detail & Related papers  (2021-02-15T06:19:25Z)
• DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks [88.62288327934499]
  We propose a novel augmentation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-11-03T07:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.