Applying LLMs to Active Learning: Towards Cost-Efficient Cross-Task Text Classification without Manually Labeled Data

• URL: http://arxiv.org/abs/2502.16892v1
• Date: Mon, 24 Feb 2025 06:43:19 GMT
• Title: Applying LLMs to Active Learning: Towards Cost-Efficient Cross-Task Text Classification without Manually Labeled Data
• Authors: Yejian Zhang, Shingo Takada,
• Abstract summary: We propose an approach that integrates large language models (LLMs) into an active learning framework.<n>Our approach achieves high cross-task text classification performance without the need for any manually labeled data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Machine learning-based classifiers have been used for text classification, such as sentiment analysis, news classification, and toxic comment classification. However, supervised machine learning models often require large amounts of labeled data for training, and manual annotation is both labor-intensive and requires domain-specific knowledge, leading to relatively high annotation costs. To address this issue, we propose an approach that integrates large language models (LLMs) into an active learning framework. Our approach combines the Robustly Optimized BERT Pretraining Approach (RoBERTa), Generative Pre-trained Transformer (GPT), and active learning, achieving high cross-task text classification performance without the need for any manually labeled data. Furthermore, compared to directly applying GPT for classification tasks, our approach retains over 93% of its classification performance while requiring only approximately 6% of the computational time and monetary cost, effectively balancing performance and resource efficiency. These findings provide new insights into the efficient utilization of LLMs and active learning algorithms in text classification tasks, paving the way for their broader application.

Related papers

• Graph-based Retrieval Augmented Generation for Dynamic Few-shot Text Classification [15.0627807767152]
  We propose a graph-based online retrieval-augmented generation framework, namely GORAG, for dynamic few-shot text classification.<n>GORAG constructs and maintains a weighted graph by extracting side information across all target texts.<n> Empirical evaluations demonstrate that GORAG outperforms existing approaches by providing more comprehensive and precise contextual information.
  arXiv  Detail & Related papers  (2025-01-06T08:43:31Z)
• Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation [2.0411082897313984]
  This study introduces a novel methodology that integrates human annotators and Large Language Models. The proposed framework integrates human annotation with the output of LLMs, depending on the model uncertainty levels. The empirical results show a substantial decrease in the costs associated with data annotation while either maintaining or improving model accuracy.
  arXiv  Detail & Related papers  (2024-06-17T21:45:48Z)
• Towards Efficient Active Learning in NLP via Pretrained Representations [1.90365714903665]
  Fine-tuning Large Language Models (LLMs) is now a common approach for text classification in a wide range of applications. We drastically expedite this process by using pretrained representations of LLMs within the active learning loop. Our strategy yields similar performance to fine-tuning all the way through the active learning loop but is orders of magnitude less computationally expensive.
  arXiv  Detail & Related papers  (2024-02-23T21:28:59Z)
• Learning to Rank for Active Learning via Multi-Task Bilevel Optimization [29.207101107965563]
  We propose a novel approach for active learning, which aims to select batches of unlabeled instances through a learned surrogate model for data acquisition. A key challenge in this approach is developing an acquisition function that generalizes well, as the history of data, which forms part of the utility function's input, grows over time.
  arXiv  Detail & Related papers  (2023-10-25T22:50:09Z)
• Prefer to Classify: Improving Text Classifiers via Auxiliary Preference Learning [76.43827771613127]
  In this paper, we investigate task-specific preferences between pairs of input texts as a new alternative way for such auxiliary data annotation. We propose a novel multi-task learning framework, called prefer-to-classify (P2C), which can enjoy the cooperative effect of learning both the given classification task and the auxiliary preferences.
  arXiv  Detail & Related papers  (2023-06-08T04:04:47Z)
• Automated Few-shot Classification with Instruction-Finetuned Language Models [76.69064714392165]
  We show that AuT-Few outperforms state-of-the-art few-shot learning methods. We also show that AuT-Few is the best ranking method across datasets on the RAFT few-shot benchmark.
  arXiv  Detail & Related papers  (2023-05-21T21:50:27Z)
• Active Learning for Abstractive Text Summarization [50.79416783266641]
  We propose the first effective query strategy for Active Learning in abstractive text summarization. We show that using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores.
  arXiv  Detail & Related papers  (2023-01-09T10:33:14Z)
• An Efficient Active Learning Pipeline for Legal Text Classification [2.462514989381979]
  We propose a pipeline for effectively using active learning with pre-trained language models in the legal domain. We use knowledge distillation to guide the model's embeddings to a semantically meaningful space. Our experiments on Contract-NLI, adapted to the classification task, and LEDGAR benchmarks show that our approach outperforms standard AL strategies.
  arXiv  Detail & Related papers  (2022-11-15T13:07:02Z)
• Assisted Text Annotation Using Active Learning to Achieve High Quality with Little Effort [9.379650501033465]
  We propose a tool that enables researchers to create large, high-quality, annotated datasets with only a few manual annotations. We combine an active learning (AL) approach with a pre-trained language model to semi-automatically identify annotation categories. Our preliminary results show that employing AL strongly reduces the number of annotations for correct classification of even complex and subtle frames.
  arXiv  Detail & Related papers  (2021-12-15T13:14:58Z)
• Knowledge-Aware Meta-learning for Low-Resource Text Classification [87.89624590579903]
  This paper studies a low-resource text classification problem and bridges the gap between meta-training and meta-testing tasks. We propose KGML to introduce additional representation for each sentence learned from the extracted sentence-specific knowledge graph.
  arXiv  Detail & Related papers  (2021-09-10T07:20:43Z)
• Semi-supervised Batch Active Learning via Bilevel Optimization [89.37476066973336]
  We formulate our approach as a data summarization problem via bilevel optimization. We show that our method is highly effective in keyword detection tasks in the regime when only few labeled samples are available.
  arXiv  Detail & Related papers  (2020-10-19T16:53:24Z)
• Self-training Improves Pre-training for Natural Language Understanding [63.78927366363178]
  We study self-training as another way to leverage unlabeled data through semi-supervised learning. We introduce SentAugment, a data augmentation method which computes task-specific query embeddings from labeled data. Our approach leads to scalable and effective self-training with improvements of up to 2.6% on standard text classification benchmarks.
  arXiv  Detail & Related papers  (2020-10-05T17:52:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.