A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs

• URL: http://arxiv.org/abs/2601.08024v1
• Date: Mon, 12 Jan 2026 21:57:33 GMT
• Title: A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs
• Abstract summary: Concept-Based Diversity (CBD) is a highly efficient metric for image inputs that leverages Vision-Language Models (VLM)<n>CBD exhibits a strong correlation with Geometric Diversity (GD), an established diversity metric, while requiring only a fraction of its time.<n>We propose a hybrid input selection approach that combines CBD with Margin, a simple uncertainty metric.
• Score: 0.09332987715848712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Maintaining or improving the performance of Deep Neural Networks (DNNs) through fine-tuning requires labeling newly collected inputs, a process that is often costly and time-consuming. To alleviate this problem, input selection approaches have been developed in recent years to identify small, yet highly informative subsets for labeling. Diversity-based selection is one of the most effective approaches for this purpose. However, they are often computationally intensive and lack scalability for large input sets, limiting their practical applicability. To address this challenge, we introduce Concept-Based Diversity (CBD), a highly efficient metric for image inputs that leverages Vision-Language Models (VLM). Our results show that CBD exhibits a strong correlation with Geometric Diversity (GD), an established diversity metric, while requiring only a fraction of its computation time. Building on this finding, we propose a hybrid input selection approach that combines CBD with Margin, a simple uncertainty metric. We conduct a comprehensive evaluation across a diverse set of DNN models, input sets, selection budgets, and five most effective state-of-the-art selection baselines. The results demonstrate that the CBD-based selection consistently outperforms all baselines at guiding input selection to improve the DNN model. Furthermore, the CBD-based selection approach remains highly efficient, requiring selection times close to those of simple uncertainty-based methods such as Margin, even on larger input sets like ImageNet. These results confirm not only the effectiveness and computational advantage of the CBD-based approach, particularly compared to hybrid baselines, but also its scalability in repetitive and extensive input selection scenarios.

Related papers

• Improving search efficiency via adaptive acquisition function selection in discrete black-box optimization [0.7734726150561088]
  In black-box optimization, it is important to identify promising solutions efficiently within a limited number of trials.<n>We propose a hybrid method that uses BOCS as the main search framework and generates alternative unevaluated points.<n>We show that its effectiveness comes from selecting points that promote search progress within Hamming-distance neighborhoods.
  arXiv  Detail & Related papers  (2026-05-11T17:03:01Z)
• Reliability-Aware Determinantal Point Processes for Robust Informative Data Selection in Large Language Models [6.463509584149428]
  We introduce ProbDPP, a reliability-aware implementation of k-DPP that accounts for probabilistic data access.<n>We frame this reliability-aware diversity as a semi-bandit problem and propose a UCB-style algorithm to efficiently learn unknown reliability online.
  arXiv  Detail & Related papers  (2026-01-31T20:14:33Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection [52.716143424856185]
  We propose LiMA (Less input is More faithful for Attribution), which reformulates the attribution of important regions as an optimization problem for submodular subset selection.<n>LiMA identifies both the most and least important samples while ensuring an optimal attribution boundary that minimizes errors.<n>Our method also outperforms the greedy search in attribution efficiency, being 1.6 times faster.
  arXiv  Detail & Related papers  (2025-04-01T06:58:15Z)
• Efficient Semi-Supervised Adversarial Training via Latent Clustering-Based Data Reduction [3.3739429143238784]
  Recent works propose semi-supervised adversarial training (SSAT) methods with external unlabeled or synthetically generated data.<n>SSAT requires substantial extra data to attain high robustness, resulting in prolonged training time and increased memory usage.<n>We propose unlabeled data reduction strategies to improve the efficiency of SSAT.
  arXiv  Detail & Related papers  (2025-01-15T15:47:49Z)
• Switchable Decision: Dynamic Neural Generation Networks [98.61113699324429]
  We propose a switchable decision to accelerate inference by dynamically assigning resources for each data instance. Our method benefits from less cost during inference while keeping the same accuracy.
  arXiv  Detail & Related papers  (2024-05-07T17:44:54Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Model-based feature selection for neural networks: A mixed-integer programming approach [0.9281671380673306]
  We develop a novel input feature selection framework for ReLU-based deep neural networks (DNNs) We focus on finding input features for image classification for clarity of presentation. We show that the proposed input feature selection allows us to drastically reduce the size of the input to $sim$15% while maintaining a good classification accuracy.
  arXiv  Detail & Related papers  (2023-02-20T22:19:50Z)
• Data-Driven Evolutionary Multi-Objective Optimization Based on Multiple-Gradient Descent for Disconnected Pareto Fronts [6.560512252982714]
  This paper proposes a data-driven evolutionary multi-objective optimization (EMO) algorithm based on multiple-gradient descent. Its infill criterion recommends a batch of promising candidate solutions to conduct expensive objective function evaluations.
  arXiv  Detail & Related papers  (2022-05-28T06:01:41Z)
• Compactness Score: A Fast Filter Method for Unsupervised Feature Selection [66.84571085643928]
  We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features. Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
  arXiv  Detail & Related papers  (2022-01-31T13:01:37Z)
• RoMA: Robust Model Adaptation for Offline Model-based Optimization [115.02677045518692]
  We consider the problem of searching an input maximizing a black-box objective function given a static dataset of input-output queries. A popular approach to solving this problem is maintaining a proxy model that approximates the true objective function. Here, the main challenge is how to avoid adversarially optimized inputs during the search.
  arXiv  Detail & Related papers  (2021-10-27T05:37:12Z)
• Sparse Bayesian Deep Learning for Dynamic System Identification [14.040914364617418]
  This paper proposes a sparse Bayesian treatment of deep neural networks (DNNs) for system identification. The proposed Bayesian approach offers a principled way to alleviate the challenges by marginal likelihood/model evidence approximation. The effectiveness of the proposed Bayesian approach is demonstrated on several linear and nonlinear systems identification benchmarks.
  arXiv  Detail & Related papers  (2021-07-27T16:09:48Z)
• Automatic selection of basis-adaptive sparse polynomial chaos expansions for engineering applications [0.0]
  We describe three state-of-the-art basis-adaptive approaches for sparse chaos expansions. We conduct an extensive benchmark in terms of global approximation accuracy on a large set of computational models. We introduce a novel solver and basis adaptivity selection scheme guided by cross-validation error.
  arXiv  Detail & Related papers  (2020-09-10T12:13:57Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.