Dataset Pruning in RecSys and ML: Best Practice or Mal-Practice?

• URL: http://arxiv.org/abs/2510.14704v1
• Date: Thu, 16 Oct 2025 14:08:30 GMT
• Title: Dataset Pruning in RecSys and ML: Best Practice or Mal-Practice?
• Abstract summary: This thesis examines the impact of data pruning on both dataset characteristics and algorithm performance.<n>Five benchmark datasets were analysed in both their unpruned form and at five successive pruning levels.<n>Results show that commonly applied core pruning can be highly selective, leaving as little as 2% of the original users in some datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Offline evaluations in recommender system research depend heavily on datasets, many of which are pruned, such as the widely used MovieLens collections. This thesis examines the impact of data pruning - specifically, removing users with fewer than a specified number of interactions - on both dataset characteristics and algorithm performance. Five benchmark datasets were analysed in both their unpruned form and at five successive pruning levels (5, 10, 20, 50, 100). For each coreset, we examined structural and distributional characteristics and trained and tested eleven representative algorithms. To further assess if pruned datasets lead to artificially inflated performance results, we also evaluated models trained on the pruned train sets but tested on unpruned data. Results show that commonly applied core pruning can be highly selective, leaving as little as 2% of the original users in some datasets. Traditional algorithms achieved higher nDCG@10 scores when both training and testing on pruned data; however, this advantage largely disappeared when evaluated on unpruned test sets. Across all algorithms, performance declined with increasing pruning levels when tested on unpruned data, highlighting the impact of dataset reduction on the performance of recommender algorithms.

Related papers

• Data Pruning: Redundant, Problematic, and Interdependent Samples [0.19336815376402716]
  We investigate two popular data pruning methods under noisy and noiseless conditions.<n>We find that these methods fail in the presence of significant label noise.<n>We highlight that the success of data pruning is distinctly affected by three factors: redundancy in the dataset, the presence of problematic samples, and interdependence between samples.
  arXiv  Detail & Related papers  (2026-06-20T07:27:23Z)
• Offline Behavioral Data Selection [58.116300485427764]
  We show that policy performance rapidly saturates when trained on a small fraction of the dataset.<n>We propose a simple yet effective method, Stepwise Dual Ranking (SDR), which extracts a compact yet informative subset from large-scale offline behavioral datasets.<n>Extensive experiments and ablation studies on D4RL benchmarks demonstrate that SDR significantly enhances data selection for offline behavioral data.
  arXiv  Detail & Related papers  (2025-12-20T07:10:58Z)
• Effective Data Pruning through Score Extrapolation [40.61665742457229]
  We introduce a novel importance score extrapolation framework that requires training on only a small subset of data.<n>We present two initial approaches in this framework to accurately predict sample importance for the entire dataset using patterns learned from this minimal subset.<n>Our results indicate that score extrapolation is a promising direction to scale expensive score calculation methods, such as pruning, data attribution, or other tasks.
  arXiv  Detail & Related papers  (2025-06-10T17:38:49Z)
• Tree Boosting Methods for Balanced andImbalanced Classification and their Robustness Over Time in Risk Assessment [0.10925516251778125]
  Tree-based methods such as XGBoost, stand out in several benchmarks due to detection performance and speed.<n>The developed method increases its recognition performance as more data is given for training.<n>It is still significantly superior to the baseline of precision-recall determined by the ratio of positives divided by positives and negatives.
  arXiv  Detail & Related papers  (2025-04-25T07:35:38Z)
• A CLIP-Powered Framework for Robust and Generalizable Data Selection [51.46695086779598]
  Real-world datasets often contain redundant and noisy data, imposing a negative impact on training efficiency and model performance.<n>Data selection has shown promise in identifying the most representative samples from the entire dataset.<n>We propose a novel CLIP-powered data selection framework that leverages multimodal information for more robust and generalizable sample selection.
  arXiv  Detail & Related papers  (2024-10-15T03:00:58Z)
• Exploring Learning Complexity for Efficient Downstream Dataset Pruning [8.990878450631596]
  Existing dataset pruning methods require training on the entire dataset.<n>We propose a straightforward, novel, and training-free hardness score named Distorting-based Learning Complexity (DLC)<n>Our method is motivated by the observation that easy samples learned faster can also be learned with fewer parameters.
  arXiv  Detail & Related papers  (2024-02-08T02:29:33Z)
• Not All Data Matters: An End-to-End Adaptive Dataset Pruning Framework for Enhancing Model Performance and Efficiency [9.460023981858319]
  We propose an end-to-end Adaptive DAtaset PRUNing framework called AdaPruner. AdaPruner iteratively prunes redundant samples to an expected pruning ratio. It can still significantly enhance model performance even after pruning up to 10-30% of the training data.
  arXiv  Detail & Related papers  (2023-12-09T16:01:21Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)
• Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics [88.39382177059747]
  A corpus of metrics is designed for measuring the accuracy, robustness, and bounds of algorithms for learning with long-tailed distribution. Based on our benchmarks, we re-evaluate the performance of existing methods on CIFAR10 and CIFAR100 datasets.
  arXiv  Detail & Related papers  (2023-02-03T02:40:54Z)
• Dataset Pruning: Reducing Training Data by Examining Generalization Influence [30.30255670341501]
  Do all training data contribute to model's performance? How to construct a smallest subset from the entire training data as a proxy training set without significantly sacrificing the model's performance?
  arXiv  Detail & Related papers  (2022-05-19T05:36:35Z)
• Doing Great at Estimating CATE? On the Neglected Assumptions in Benchmark Comparisons of Treatment Effect Estimators [91.3755431537592]
  We show that even in arguably the simplest setting, estimation under ignorability assumptions can be misleading. We consider two popular machine learning benchmark datasets for evaluation of heterogeneous treatment effect estimators. We highlight that the inherent characteristics of the benchmark datasets favor some algorithms over others.
  arXiv  Detail & Related papers  (2021-07-28T13:21:27Z)
• Self-Trained One-class Classification for Unsupervised Anomaly Detection [56.35424872736276]
  Anomaly detection (AD) has various applications across domains, from manufacturing to healthcare. In this work, we focus on unsupervised AD problems whose entire training data are unlabeled and may contain both normal and anomalous samples. To tackle this problem, we build a robust one-class classification framework via data refinement. We show that our method outperforms state-of-the-art one-class classification method by 6.3 AUC and 12.5 average precision.
  arXiv  Detail & Related papers  (2021-06-11T01:36:08Z)
• Outlier Guided Optimization of Abdominal Segmentation [7.036733782879497]
  We build on a pre-trained 3D U-Net model for abdominal multi-organ segmentation. We augmented the dataset either with outlier data (e.g., exemplars for which the baseline algorithm failed) or inliers (e.g., exemplars for which the baseline algorithm worked) We find that the marginal value of adding outliers is higher than that of adding inliers.
  arXiv  Detail & Related papers  (2020-02-10T21:41:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.