CTDGSI: A comprehensive exploitation of instance selection methods for automatic text classification. VII Concurso de Teses, Dissertações e Trabalhos de Graduação em SI -- XXI Simpósio Brasileiro de Sistemas de Informação

• URL: http://arxiv.org/abs/2506.07169v1
• Date: Sun, 08 Jun 2025 14:34:57 GMT
• Title: CTDGSI: A comprehensive exploitation of instance selection methods for automatic text classification. VII Concurso de Teses, Dissertações e Trabalhos de Graduação em SI -- XXI Simpósio Brasileiro de Sistemas de Informação
• Abstract summary: This dissertation focuses on an under-investi-gated NLP data engineering technique, known as Instance Selection (IS)<n>The IS goal is to reduce the training set size by removing noisy or redundant instances while maintaining the effectiveness of the trained models.<n>Our solutions demonstrated speedup improvements of 1.67x (up to 2.46x), making them scalable for datasets with hundreds of thousands of documents.
• Score: 9.069939079065298
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Progress in Natural Language Processing (NLP) has been dictated by the rule of more: more data, more computing power and more complexity, best exemplified by the Large Language Models. However, training (or fine-tuning) large dense models for specific applications usually requires significant amounts of computing resources. This \textbf{Ph.D. dissertation} focuses on an under-investi\-gated NLP data engineering technique, whose potential is enormous in the current scenario known as Instance Selection (IS). The IS goal is to reduce the training set size by removing noisy or redundant instances while maintaining the effectiveness of the trained models and reducing the training process cost. We provide a comprehensive and scientifically sound comparison of IS methods applied to an essential NLP task -- Automatic Text Classification (ATC), considering several classification solutions and many datasets. Our findings reveal a significant untapped potential for IS solutions. We also propose two novel IS solutions that are noise-oriented and redundancy-aware, specifically designed for large datasets and transformer architectures. Our final solution achieved an average reduction of 41\% in training sets, while maintaining the same levels of effectiveness in all datasets. Importantly, our solutions demonstrated speedup improvements of 1.67x (up to 2.46x), making them scalable for datasets with hundreds of thousands of documents.

Related papers

• PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning [6.950650586528147]
  We propose PPL-Factory, a simple and interpretable data selection framework.<n>Experiments on GSM8K demonstrate that PPL-Factory outperforms other state-of-the-art data selection methods.
  arXiv  Detail & Related papers  (2026-07-20T17:38:38Z)
• Entropy-Based Data Selection for Language Models [12.922021171941216]
  Modern language models (LMs) increasingly require two critical resources: computational resources and data resources.<n>Data selection techniques can effectively reduce the amount of training data required for fine-tuning LMs.<n>We propose Entropy-Based Unsupervised Data Selection (EUDS) framework for efficient data selection.
  arXiv  Detail & Related papers  (2026-02-19T15:29:34Z)
• Oversampling and Downsampling with Core-Boundary Awareness: A Data Quality-Driven Approach [2.334306891078381]
  We propose a method to systematically identify and differentiate between two types of data.<n>By prioritizing high-quality, decision-relevant data, our approach can be extended to text, multimodal, and self-supervised learning scenarios.<n>This work paves the way for future research in data-efficient learning, where intelligent sampling replaces brute-force expansion.
  arXiv  Detail & Related papers  (2025-09-24T07:55:07Z)
• Does This Look Familiar to You? Knowledge Analysis via Model Internal Representations [0.0]
  There is no clearly established methodology for effective training data selection.<n>Model Internal Representations (KAMIR) is a novel approach that overcomes these limitations.<n>It can be applied to a wide range of tasks such as machine reading comprehension and summarization.
  arXiv  Detail & Related papers  (2025-09-09T01:08:15Z)
• SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models [51.74498855100541]
  Large language models (LLMs) have shown strong reasoning capabilities when fine-tuned with reinforcement learning (RL)<n>We propose textbfSPaRFT, a self-paced learning framework that enables efficient learning based on the capability of the model being trained.
  arXiv  Detail & Related papers  (2025-08-07T03:50:48Z)
• Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Framework [81.29965270493238]
  We develop a specialized dataset aimed at enhancing the evaluation and fine-tuning of large language models (LLMs) for wireless communication applications.<n>The dataset includes a diverse set of multi-hop questions, including true/false and multiple-choice types, spanning varying difficulty levels from easy to hard.<n>We introduce a Pointwise V-Information (PVI) based fine-tuning method, providing a detailed theoretical analysis and justification for its use in quantifying the information content of training data.
  arXiv  Detail & Related papers  (2025-01-16T16:19:53Z)
• YuLan-Mini: An Open Data-efficient Language Model [111.02822724500552]
  YuLan-Mini, a highly capable base model with 2.42B parameters, achieves top-tier performance among models of similar parameter scale.<n>Remarkably, YuLan-Mini, trained on 1.08T tokens, achieves performance comparable to industry-leading models that require significantly more data.
  arXiv  Detail & Related papers  (2024-12-23T17:47:53Z)
• Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratch [54.12139707822201]
  We propose ScaleQuest, a novel, scalable, and cost-effective data synthesis method.<n>By generating diverse questions from scratch, we produce a dataset of 1 million problem-solution pairs.<n>Our experiments demonstrate that models trained on our data outperform existing open-source datasets.
  arXiv  Detail & Related papers  (2024-10-24T12:42:04Z)
• Unsupervised Data Validation Methods for Efficient Model Training [0.0]
  State-of-the-art models in natural language processing (NLP), text-to-speech (TTS), speech-to-text (STT) and vision-language models (VLM) rely heavily on large datasets. This research explores key areas such as defining "quality data," developing methods for generating appropriate data and enhancing accessibility to model training.
  arXiv  Detail & Related papers  (2024-10-10T13:00:53Z)
• Improving Classifier Training Efficiency for Automatic Cyberbullying Detection with Feature Density [58.64907136562178]
  We study the effectiveness of Feature Density (FD) using different linguistically-backed feature preprocessing methods. We hypothesise that estimating dataset complexity allows for the reduction of the number of required experiments. The difference in linguistic complexity of datasets allows us to additionally discuss the efficacy of linguistically-backed word preprocessing.
  arXiv  Detail & Related papers  (2021-11-02T15:48:28Z)
• DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference [86.03382625531951]
  DANCE is an automated simultaneous data-network co-optimization for efficient segmentation model training and inference.<n>It integrates automated data slimming which adaptively downsamples/drops input images and controls their corresponding contribution to the training loss guided by the images' spatial complexity.<n>Experiments and ablating studies demonstrate that DANCE can achieve "all-win" towards efficient segmentation.
  arXiv  Detail & Related papers  (2021-07-16T04:58:58Z)
• Training Data Subset Selection for Regression with Controlled Generalization Error [19.21682938684508]
  We develop an efficient majorization-minimization algorithm for data subset selection. SELCON trades off accuracy and efficiency more effectively than the current state-of-the-art.
  arXiv  Detail & Related papers  (2021-06-23T16:03:55Z)
• GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning [11.220278271829699]
  We introduce Glister, a GeneraLIzation based data Subset selecTion for Efficient and Robust learning framework. We propose an iterative online algorithm Glister-Online, which performs data selection iteratively along with the parameter updates. We show that our framework improves upon state of the art both in efficiency and accuracy (in cases (a) and (c)) and is more efficient compared to other state-of-the-art robust learning algorithms.
  arXiv  Detail & Related papers  (2020-12-19T08:41:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.