Data Pruning: Redundant, Problematic, and Interdependent Samples

• URL: http://arxiv.org/abs/2606.21916v1
• Date: Sat, 20 Jun 2026 07:27:23 GMT
• Title: Data Pruning: Redundant, Problematic, and Interdependent Samples
• Abstract summary: We investigate two popular data pruning methods under noisy and noiseless conditions.<n>We find that these methods fail in the presence of significant label noise.<n>We highlight that the success of data pruning is distinctly affected by three factors: redundancy in the dataset, the presence of problematic samples, and interdependence between samples.
• Score: 0.19336815376402716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The performance of deep learning models is affected by not only data quantity but also data quality. Data pruning is a process by which practitioners can reduce the size of a dataset by only keeping the most important training data points, thereby achieving similar test set performance. We empirically investigate two popular data pruning methods under noisy and noiseless conditions and show that these methods fail in the presence of significant label noise. We highlight that the success of data pruning is distinctly affected by three factors: redundancy in the dataset, the presence of problematic samples, and interdependence between samples. We perform a detailed investigation on commonly used benchmark classification datasets and neural network architectures. We find that our observations are consistent across data distributions and training protocols.

Related papers

• Understanding Data Influence with Differential Approximation [63.817689230826595]
  We introduce a new formulation to approximate a sample's influence by accumulating the differences in influence between consecutive learning steps, which we term Diff-In.<n>By employing second-order approximations, we approximate these difference terms with high accuracy while eliminating the need for model convexity required by existing methods.<n>Our theoretical analysis demonstrates that Diff-In achieves significantly lower approximation error compared to existing influence estimators.
  arXiv  Detail & Related papers  (2025-08-20T11:59:32Z)
• Navigating Data Corruption in Machine Learning: Balancing Quality, Quantity, and Imputation Strategies [8.770864706004472]
  Data corruption, including missing and noisy data, poses significant challenges in real-world machine learning.<n>This study investigates the effects of data corruption on model performance and explores strategies to mitigate these effects.<n>We find that increasing dataset size mitigates but cannot fully overcome the effects of data corruption.
  arXiv  Detail & Related papers  (2024-12-24T09:04:06Z)
• Data Pruning Can Do More: A Comprehensive Data Pruning Approach for Object Re-identification [13.732596789612362]
  This work is the first to explore the feasibility of data pruning methods applied to object re-identification tasks.<n>By fully leveraging the logit history during training, our approach offers a more accurate and comprehensive metric for quantifying sample importance.<n>Our approach is highly efficient, reducing the cost of importance score estimation by 10 times compared to existing methods.
  arXiv  Detail & Related papers  (2024-12-13T12:27:47Z)
• Robust Noisy Correspondence Learning via Self-Drop and Dual-Weight [11.523154025649758]
  Crowd-sourcing or web crawling introduces mismatched pairs.<n>Current approaches leverage the effect of deep neural networks to distinguish noise and perform re-weighting.<n>We propose a novel self-drop and dual-weight approach, which achieves elaborate data processing by qua- Partitioning the data.
  arXiv  Detail & Related papers  (2024-12-09T03:06:10Z)
• A CLIP-Powered Framework for Robust and Generalizable Data Selection [51.46695086779598]
  Real-world datasets often contain redundant and noisy data, imposing a negative impact on training efficiency and model performance.<n>Data selection has shown promise in identifying the most representative samples from the entire dataset.<n>We propose a novel CLIP-powered data selection framework that leverages multimodal information for more robust and generalizable sample selection.
  arXiv  Detail & Related papers  (2024-10-15T03:00:58Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Benchmark Transparency: Measuring the Impact of Data on Evaluation [6.307485015636125]
  We propose an automated framework that measures the data point distribution across 6 different dimensions. We use disproportional stratified sampling to measure how much the data distribution affects absolute (Acc/F1) and relative (Rank) model performance. We find that the impact of the data is statistically significant and is often larger than the impact of changing the metric.
  arXiv  Detail & Related papers  (2024-03-31T17:33:43Z)
• Combining Observational and Randomized Data for Estimating Heterogeneous Treatment Effects [82.20189909620899]
  Estimating heterogeneous treatment effects is an important problem across many domains. Currently, most existing works rely exclusively on observational data. We propose to estimate heterogeneous treatment effects by combining large amounts of observational data and small amounts of randomized data.
  arXiv  Detail & Related papers  (2022-02-25T18:59:54Z)
• Assessing the Quality of the Datasets by Identifying Mislabeled Samples [14.881597737762316]
  We propose a novel statistic -- noise score -- as a measure for the quality of each data point to identify mislabeled samples. In our work, we use the representations derived by the inference network of data quality supervised variational autoencoder (AQUAVS) We validate our proposed statistic through experimentation by corrupting MNIST, FashionMNIST, and CIFAR10/100 datasets.
  arXiv  Detail & Related papers  (2021-09-10T17:14:09Z)
• Improving Multi-Turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting [84.9716460244444]
  We consider utilizing the underlying correlation in the data resource itself to derive different kinds of supervision signals. We conduct extensive experiments in two public datasets and obtain significant improvement in both datasets.
  arXiv  Detail & Related papers  (2020-02-18T06:29:01Z)
• Learning with Out-of-Distribution Data for Audio Classification [60.48251022280506]
  We show that detecting and relabelling certain OOD instances, rather than discarding them, can have a positive effect on learning. The proposed method is shown to improve the performance of convolutional neural networks by a significant margin.
  arXiv  Detail & Related papers  (2020-02-11T21:08:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.