Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Method

• URL: http://arxiv.org/abs/2510.21286v1
• Date: Fri, 24 Oct 2025 09:33:19 GMT
• Title: Adaptive Data Selection for Multi-Layer Perceptron Training: A Sub-linear Value-Driven Method
• Abstract summary: Data selection is one of the fundamental problems in neural network training.<n>Data Value Contribution is a novel budget-aware method for selecting data for training.<n>Our method consistently outperforms existing approaches under various budget constraints.
• Score: 10.014138113199616
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data selection is one of the fundamental problems in neural network training, particularly for multi-layer perceptrons (MLPs) where identifying the most valuable training samples from massive, multi-source, and heterogeneous data sources under budget constraints poses significant challenges. Existing data selection methods, including coreset construction, data Shapley values, and influence functions, suffer from critical limitations: they oversimplify nonlinear transformations, ignore informative intermediate representations in hidden layers, or fail to scale to larger MLPs due to high computational complexity. In response, we propose DVC (Data Value Contribution), a novel budget-aware method for evaluating and selecting data for MLP training that accounts for the dynamic evolution of network parameters during training. The DVC method decomposes data contribution into Layer Value Contribution (LVC) and Global Value Contribution (GVC), employing six carefully designed metrics and corresponding efficient algorithms to capture data characteristics across three dimensions--quality, relevance, and distributional diversity--at different granularities. DVC integrates these assessments with an Upper Confidence Bound (UCB) algorithm for adaptive source selection that balances exploration and exploitation. Extensive experiments across six datasets and eight baselines demonstrate that our method consistently outperforms existing approaches under various budget constraints, achieving superior accuracy and F1 scores. Our approach represents the first systematic treatment of hierarchical data evaluation for neural networks, providing both theoretical guarantees and practical advantages for large-scale machine learning systems.

Related papers

• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• Data-Efficient RLVR via Off-Policy Influence Guidance [84.60336960383867]
  This work proposes a theoretically-grounded approach using influence functions to estimate the contribution of each data point to the learning objective.<n>We develop textbfCurriculum textbfRL with textbfOff-textbfPolicy textInfluence guidance (textbfCROPI), a multi-stage RL framework that iteratively selects the most influential data for the current policy.
  arXiv  Detail & Related papers  (2025-10-30T13:40:52Z)
• Oversampling and Downsampling with Core-Boundary Awareness: A Data Quality-Driven Approach [2.334306891078381]
  We propose a method to systematically identify and differentiate between two types of data.<n>By prioritizing high-quality, decision-relevant data, our approach can be extended to text, multimodal, and self-supervised learning scenarios.<n>This work paves the way for future research in data-efficient learning, where intelligent sampling replaces brute-force expansion.
  arXiv  Detail & Related papers  (2025-09-24T07:55:07Z)
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward [54.708851958671794]
  We propose a Data-Efficient Policy Optimization pipeline that combines optimized strategies for both offline and online data selection.<n>In offline phase, we curate a high-quality subset of training samples based on diversity, influence, and appropriate difficulty.<n>During online RLVR training, we introduce a sample-level explorability metric to dynamically filter samples with low exploration potential.
  arXiv  Detail & Related papers  (2025-09-01T10:04:20Z)
• Federated Online Learning for Heterogeneous Multisource Streaming Data [0.0]
  Federated learning has emerged as an essential paradigm for distributed multi-source data analysis under privacy concerns.<n>In this paper, we propose a federated online learning (FOL) method for distributed multi-source streaming data analysis.
  arXiv  Detail & Related papers  (2025-08-08T19:08:53Z)
• Selective Embedding for Deep Learning [0.4499833362998489]
  Deep learning algorithms are sensitive to input data, and performance often deteriorates under nonstationary conditions.<n>This study introduces selective embedding, a novel data loading strategy, which alternates short segments of data from multiple sources within a single input channel.
  arXiv  Detail & Related papers  (2025-07-16T15:45:01Z)
• DACL-RAG: Data Augmentation Strategy with Curriculum Learning for Retrieval-Augmented Generation [54.26665681604041]
  We introduce DACL-RAG, a multi-stage RAG training framework that combines a multi-level Data Augmentation strategy and a multi-stage Curriculum Learning paradigm.<n>Our framework demonstrates consistent effectiveness across four open-domain QA datasets, achieving performance gains of 2% to 4% over multiple advanced methods.
  arXiv  Detail & Related papers  (2025-05-15T16:53:04Z)
• Adaptive Dataset Quantization [2.0105434963031463]
  We introduce a versatile framework for dataset compression, namely Adaptive dataset Quantization (ADQ)<n>We propose a novel adaptive sampling strategy through the evaluation of generated bins' representativeness score, diversity score and importance score.<n>Our method not only exhibits superior generalization capability across different architectures, but also attains state-of-the-art results, surpassing DQ by average 3% on various datasets.
  arXiv  Detail & Related papers  (2024-12-22T07:08:29Z)
• Leveraging Semi-Supervised Learning to Enhance Data Mining for Image Classification under Limited Labeled Data [35.431340001608476]
  Traditional data mining methods are inadequate when faced with large-scale, high-dimensional and complex data.<n>This study introduces semi-supervised learning methods, aiming to improve the algorithm's ability to utilize unlabeled data.<n> Specifically, we adopt a self-training method and combine it with a convolutional neural network (CNN) for image feature extraction and classification.
  arXiv  Detail & Related papers  (2024-11-27T18:59:50Z)
• Robust Learning with Progressive Data Expansion Against Spurious Correlation [65.83104529677234]
  We study the learning process of a two-layer nonlinear convolutional neural network in the presence of spurious features. Our analysis suggests that imbalanced data groups and easily learnable spurious features can lead to the dominance of spurious features during the learning process. We propose a new training algorithm called PDE that efficiently enhances the model's robustness for a better worst-group performance.
  arXiv  Detail & Related papers  (2023-06-08T05:44:06Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data [77.88594632644347]
  Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization challenge. We propose a novel momentum-based method to mitigate this decentralized training difficulty.
  arXiv  Detail & Related papers  (2021-02-09T11:27:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.