GLOBE: Trajectory-Aligned Gradient Matching with Structured SparseOptimization for Coreset Selection

• URL: http://arxiv.org/abs/2608.02690v1
• Date: Mon, 03 Aug 2026 09:01:29 GMT
• Title: GLOBE: Trajectory-Aligned Gradient Matching with Structured SparseOptimization for Coreset Selection
• Abstract summary: On-device training of deep neural networks is constrained by the computational and memory costs of large-scale datasets.<n>We propose GLOBE, a trajectory-aligned coreset selection framework that formulates sample selection as a globally optimized sparse weighting problem.
• Score: 4.969458805037944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: On-device training of deep neural networks is fundamentally constrained by the computational and memory costs of large-scale datasets. Coreset selection offers a practical solution by retaining only a compact subset of real training samples. However, existing gradient-based methods commonly rely on gradients computed at a single model snapshot and employ greedy or pursuit-based selection procedures, limiting their ability to capture evolving optimization dynamics and handle strongly correlated samples. We propose GLOBE (Gradient Local-Balanced Extraction), a trajectory-aligned coreset selection framework that formulates sample selection as a globally optimized sparse weighting problem. GLOBE represents each sample by a gradient trajectory constructed across multiple training checkpoints, thereby capturing its influence throughout different stages of optimization. To preserve the training behavior of the full dataset, we introduce a multi-order matching objective that jointly aligns the first-order mean and projected uncentered second-order moments of gradient trajectories. GLOBE further combines Group LASSO, Elastic Net regularization, and nonnegative budget constraints to induce group- and sample-level sparsity while stabilizing the weights of correlated trajectories. Finally, class-balanced Top-K selection maintains adequate category coverage under limited sampling budgets. Experiments across six benchmarks and five evaluation architectures demonstrate that GLOBE consistently outperforms existing coreset selection methods in downstream test accuracy, particularly at low retention ratios. These results highlight the effectiveness of combining dynamic gradient information, multi-order distribution matching, and structured sparsity for data-efficient learning.

Related papers

• Active Learning with Low-Rank Structure for Data Selection [85.43185363043362]
  We introduce a new data selection framework based on low-rank approximation and residual-based sampling.<n>We show that it is possible to select a weighted subset of $tildeOleft(k + frac1varepsilon2right)$ data points whose average loss approximates the average loss over the full dataset.
  arXiv  Detail & Related papers  (2026-06-14T22:29:59Z)
• CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing [88.9067184995168]
  We propose a unified framework CogniEdit, combining multi-modal reasoning with dense reward optimization.<n>Our method achieves state-of-the-art performance in balancing fine-grained instruction following with visual quality and editability preservation.
  arXiv  Detail & Related papers  (2025-12-15T12:36:50Z)
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation [109.13471554184554]
  We reformulate dataset distillation as an Optimal Transport (OT) distance minimization problem.<n>OT offers a geometrically faithful framework for distribution matching.<n>Our method consistently outperforms state-of-the-art approaches in an efficient manner.
  arXiv  Detail & Related papers  (2025-11-29T04:04:05Z)
• ClusterUCB: Efficient Gradient-Based Data Selection for Targeted Fine-Tuning of LLMs [22.68421108740517]
  We propose an efficient gradient-based data selection framework with clustering and a modified Upper Confidence Bound (UCB) algorithm.<n> Experimental results on various benchmarks show that our proposed framework, ClusterUCB, can achieve comparable results to the original gradient-based data selection methods.
  arXiv  Detail & Related papers  (2025-06-12T01:53:01Z)
• Taming LLMs by Scaling Learning Rates with Gradient Grouping [49.91587150497186]
  Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures.<n>This work introduces Scaling with Gradient Grouping (SGG), an gradient wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling.
  arXiv  Detail & Related papers  (2025-06-01T15:30:37Z)
• Semantic-Aligned Learning with Collaborative Refinement for Unsupervised VI-ReID [82.12123628480371]
  Unsupervised person re-identification (USL-VI-ReID) seeks to match pedestrian images of the same individual across different modalities without human annotations for model learning.<n>Previous methods unify pseudo-labels of cross-modality images through label association algorithms and then design contrastive learning framework for global feature learning.<n>We propose a Semantic-Aligned Learning with Collaborative Refinement (SALCR) framework, which builds up objective for specific fine-grained patterns emphasized by each modality.
  arXiv  Detail & Related papers  (2025-04-27T13:58:12Z)
• Non-Uniform Class-Wise Coreset Selection: Characterizing Category Difficulty for Data-Efficient Transfer Learning [19.152700266277247]
  Non-Uniform Class-Wise Coreset Selection (NUCS) is a novel framework that integrates both class-level and instance-level criteria.<n>Our work highlights the importance of characterizing category difficulty in coreset selection, offering a robust and data-efficient solution for transfer learning.
  arXiv  Detail & Related papers  (2025-04-17T15:40:51Z)
• Coreset Selection via LLM-based Concept Bottlenecks [6.857632954159568]
  Coreset Selection (CS) aims to identify a subset of the training dataset that achieves model performance comparable to using the entire dataset.<n>Our work proposes a score that computes a sample's difficulty using human-understandable textual attributes (concepts) independent of any downstream model.<n>We show that our coresets outperform random subsets, even at high pruning rates, and achieve model performance comparable to or better than coresets found by training dynamics-based methods.
  arXiv  Detail & Related papers  (2025-02-23T22:14:42Z)
• BWS: Best Window Selection Based on Sample Scores for Data Pruning across Broad Ranges [12.248397169100784]
  Data subset selection aims to find a smaller yet informative subset of a large dataset that can approximate the full-dataset training. We introduce a universal and efficient data subset selection method, Best Window Selection (BWS), by proposing a method to choose the best window subset from samples ordered based on their difficulty scores.
  arXiv  Detail & Related papers  (2024-06-05T08:33:09Z)
• Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints [69.27190330994635]
  Coreset selection is powerful in reducing computational costs and accelerating data processing for deep learning algorithms. We propose an innovative method, which maintains optimization priority order over the model performance and coreset size. Empirically, extensive experiments confirm its superiority, often yielding better model performance with smaller coreset sizes.
  arXiv  Detail & Related papers  (2023-11-15T03:43:04Z)
• Probabilistic Bilevel Coreset Selection [24.874967723659022]
  We propose a continuous probabilistic bilevel formulation of coreset selection by learning a probablistic weight for each training sample. We develop an efficient solver to the bilevel optimization problem via unbiased policy gradient without trouble of implicit differentiation.
  arXiv  Detail & Related papers  (2023-01-24T09:37:00Z)
• Towards Automated Imbalanced Learning with Deep Hierarchical Reinforcement Learning [57.163525407022966]
  Imbalanced learning is a fundamental challenge in data mining, where there is a disproportionate ratio of training samples in each class. Over-sampling is an effective technique to tackle imbalanced learning through generating synthetic samples for the minority class. We propose AutoSMOTE, an automated over-sampling algorithm that can jointly optimize different levels of decisions.
  arXiv  Detail & Related papers  (2022-08-26T04:28:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.