Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning

• URL: http://arxiv.org/abs/2502.18978v3
• Date: Sat, 08 Mar 2025 09:47:20 GMT
• Title: Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning
• Authors: Hongyi Cai, Jie Li, Wenzhen Dong,
• Abstract summary: Low-Confidence Gold (LCG) is a novel filtering framework that employs centroid-based clustering and confidence-guided selection.<n> LCG curates high-quality subsets while preserving data diversity.<n>Models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods.
• Score: 4.24565587746027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The effectiveness of instruction fine-tuning for Large Language Models is fundamentally constrained by the quality and efficiency of training datasets. This work introduces Low-Confidence Gold (LCG), a novel filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs. Through a semi-supervised approach using a lightweight classifier trained on representative samples, LCG curates high-quality subsets while preserving data diversity. Experimental evaluation demonstrates that models fine-tuned on LCG-filtered subsets of 6K samples achieve superior performance compared to existing methods, with substantial improvements on MT-bench and consistent gains across comprehensive evaluation metrics. The framework's efficacy while maintaining model performance establishes a promising direction for efficient instruction tuning.

Related papers

• Improving Model Classification by Optimizing the Training Dataset [3.987352341101438]
  Coresets offer a principled approach to data reduction, enabling efficient learning on large datasets.<n>We present a systematic framework for tuning the coreset generation process to enhance downstream classification quality.
  arXiv  Detail & Related papers  (2025-07-22T16:10:11Z)
• LGAI-EMBEDDING-Preview Technical Report [41.68404082385825]
  This report presents a unified instruction-based framework for learning generalized text embeddings optimized for both information retrieval (IR) and non-IR tasks.<n>Our approach combines in-context learning, soft supervision, and adaptive hard-negative mining to generate context-aware embeddings.<n>Results show that our method achieves strong generalization and ranks among the top-performing models by Borda score.
  arXiv  Detail & Related papers  (2025-06-09T05:30:35Z)
• Feedback-Driven Pseudo-Label Reliability Assessment: Redefining Thresholding for Semi-Supervised Semantic Segmentation [5.7977777220041204]
  A common practice in pseudo-supervision is filtering pseudo-labels based on pre-defined confidence thresholds or entropy.<n>We propose Ensemble-of-Confidence Reinforcement (ENCORE), a dynamic feedback-driven thresholding strategy for pseudo-label selection.<n>Our method seamlessly integrates into existing pseudo-supervision frameworks and significantly improves segmentation performance.
  arXiv  Detail & Related papers  (2025-05-12T15:58:08Z)
• Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMs [15.806503459642665]
  We propose a new algorithm for fine-tuning large language models using reinforcement learning. We show that properly leveraging positive and negative examples alike in the off-policy regime simultaneously increases test-time accuracy and training data efficiency. As a corollary to this work, we find that REINFORCE's baseline parameter plays an important and unexpected role in defining dataset composition in the presence of negative examples.
  arXiv  Detail & Related papers  (2025-03-18T14:23:37Z)
• From Captions to Rewards (CAREVL): Leveraging Large Language Model Experts for Enhanced Reward Modeling in Large Vision-Language Models [58.16075709485292]
  CAREVL is a novel method for preference reward modeling by reliably using both high- and low-confidence data. CAREVL achieves performance improvements over traditional distillation-based methods on VL-RewardBench and MLLM-as-a-Judge benchmark.
  arXiv  Detail & Related papers  (2025-03-08T16:13:18Z)
• Automatic Evaluation for Text-to-image Generation: Task-decomposed Framework, Distilled Training, and Meta-evaluation Benchmark [62.58869921806019]
  We propose a task decomposition evaluation framework based on GPT-4o to automatically construct a new training dataset. We design innovative training strategies to effectively distill GPT-4o's evaluation capabilities into a 7B open-source MLLM, MiniCPM-V-2.6. Experimental results demonstrate that our distilled open-source MLLM significantly outperforms the current state-of-the-art GPT-4o-base baseline.
  arXiv  Detail & Related papers  (2024-11-23T08:06:06Z)
• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning [71.2981957820888]
  We propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets. The framework initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. The generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality.
  arXiv  Detail & Related papers  (2024-11-21T02:30:53Z)
• Self-DenseMobileNet: A Robust Framework for Lung Nodule Classification using Self-ONN and Stacking-based Meta-Classifier [1.2300841481611335]
  Self-DenseMobileNet is designed to enhance the classification of nodules and non-nodules in chest radiographs (CXRs) Our framework integrates advanced image standardization and enhancement techniques to optimize the input quality. When tested on an external dataset, the framework maintained strong generalizability with an accuracy of 89.40%.
  arXiv  Detail & Related papers  (2024-10-16T14:04:06Z)
• Language Model Preference Evaluation with Multiple Weak Evaluators [78.53743237977677]
  GED (Preference Graph Ensemble and Denoise) is a novel approach that leverages multiple model-based evaluators to construct preference graphs.<n>In particular, our method consists of two primary stages: aggregating evaluations into a unified graph and applying a denoising process.<n>We provide theoretical guarantees for our framework, demonstrating its efficacy in recovering the ground truth preference structure.
  arXiv  Detail & Related papers  (2024-10-14T01:57:25Z)
• Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
  We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios. We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-06-07T11:37:45Z)
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
  We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process. We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals. The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
  arXiv  Detail & Related papers  (2024-05-01T11:10:24Z)
• Entailment as Robust Self-Learner [14.86757876218415]
  We design a prompting strategy that formulates a number of different NLU tasks as contextual entailment. We propose the Simple Pseudo-Label Editing (SimPLE) algorithm for better pseudo-labeling quality in self-training.
  arXiv  Detail & Related papers  (2023-05-26T18:41:23Z)
• ESimCSE Unsupervised Contrastive Learning Jointly with UDA Semi-Supervised Learning for Large Label System Text Classification Mode [4.708633772366381]
  The ESimCSE model efficiently learns text vector representations using unlabeled data to achieve better classification results. UDA is trained using unlabeled data through semi-supervised learning methods to improve the prediction performance of the models and stability. adversarial training techniques FGM and PGD are used in the model training process to improve the robustness and reliability of the model.
  arXiv  Detail & Related papers  (2023-04-19T03:44:23Z)
• ENInst: Enhancing Weakly-supervised Low-shot Instance Segmentation [23.621454800084724]
  We address a weakly-supervised low-shot instance segmentation, an annotation-efficient training method to deal with novel classes effectively. Our ENInst is 7.5 times more efficient in achieving comparable performance to the existing fully-supervised few-shot models and even outperforms them at times.
  arXiv  Detail & Related papers  (2023-02-20T05:15:23Z)
• Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic [67.00475077281212]
  Model-based reinforcement learning algorithms are more sample efficient than their model-free counterparts. We propose a novel approach that achieves high sample efficiency without the strong reliance on accurate learned models. We show that CMBAC significantly outperforms state-of-the-art approaches in terms of sample efficiency on several challenging tasks.
  arXiv  Detail & Related papers  (2021-12-16T15:33:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.