Interactive Multi-fidelity Learning for Cost-effective Adaptation of
Language Model with Sparse Human Supervision
- URL: http://arxiv.org/abs/2310.20153v1
- Date: Tue, 31 Oct 2023 03:39:23 GMT
- Title: Interactive Multi-fidelity Learning for Cost-effective Adaptation of
Language Model with Sparse Human Supervision
- Authors: Jiaxin Zhang, Zhuohang Li, Kamalika Das, Sricharan Kumar
- Abstract summary: Large language models (LLMs) have demonstrated remarkable capabilities in various tasks.
We propose a novel Interactive Multi-Fidelity Learning (IMFL) framework for the cost-effective development of small domain-specific LMs.
- Score: 6.151133144093847
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in
various tasks. However, their suitability for domain-specific tasks, is limited
due to their immense scale at deployment, susceptibility to misinformation, and
more importantly, high data annotation costs. We propose a novel Interactive
Multi-Fidelity Learning (IMFL) framework for the cost-effective development of
small domain-specific LMs under limited annotation budgets. Our approach
formulates the domain-specific fine-tuning process as a multi-fidelity learning
problem, focusing on identifying the optimal acquisition strategy that balances
between low-fidelity automatic LLM annotations and high-fidelity human
annotations to maximize model performance. We further propose an
exploration-exploitation query strategy that enhances annotation diversity and
informativeness, incorporating two innovative designs: 1) prompt retrieval that
selects in-context examples from human-annotated samples to improve LLM
annotation, and 2) variable batch size that controls the order for choosing
each fidelity to facilitate knowledge distillation, ultimately enhancing
annotation quality. Extensive experiments on financial and medical tasks
demonstrate that IMFL achieves superior performance compared with single
fidelity annotations. Given a limited budget of human annotation, IMFL
significantly outperforms the human annotation baselines in all four tasks and
achieves very close performance as human annotations on two of the tasks. These
promising results suggest that the high human annotation costs in
domain-specific tasks can be significantly reduced by employing IMFL, which
utilizes fewer human annotations, supplemented with cheaper and faster LLM
(e.g., GPT-3.5) annotations to achieve comparable performance.
Related papers
- Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning [50.73666458313015]
Large Language Models (LLMs) have demonstrated significant potential in performing multiple tasks in multimedia applications.
MoE has been emerged as a promising solution with its sparse architecture for effective task decoupling.
Intuition-MoR1E achieves superior efficiency and 2.15% overall accuracy improvement across 14 public datasets.
arXiv Detail & Related papers (2024-04-13T12:14:58Z) - LLMs in the Loop: Leveraging Large Language Model Annotations for Active Learning in Low-Resource Languages [1.149936119867417]
Low-resource languages face significant barriers in AI development due to limited linguistic resources and expertise for data labeling.
We propose leveraging the potential of LLMs in the active learning loop for data annotation.
Empirical evaluations, notably employing GPT-4-Turbo, demonstrate near-state-of-the-art performance with significantly reduced data requirements.
arXiv Detail & Related papers (2024-04-02T19:34:22Z) - FAC$^2$E: Better Understanding Large Language Model Capabilities by
Dissociating Language and Cognition [57.747888532651]
Large language models (LLMs) are primarily evaluated by overall performance on various text understanding and generation tasks.
We present FAC$2$E, a framework for Fine-grAined and Cognition-grounded LLMs' Capability Evaluation.
arXiv Detail & Related papers (2024-02-29T21:05:37Z) - An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models [55.01592097059969]
Supervised finetuning on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities.
Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool.
We propose using experimental design to circumvent the computational bottlenecks of active learning.
arXiv Detail & Related papers (2024-01-12T16:56:54Z) - Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering.
The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored.
We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
arXiv Detail & Related papers (2023-12-26T07:24:46Z) - CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large
Language Models for Data Annotation [94.59630161324013]
We propose CoAnnotating, a novel paradigm for Human-LLM co-annotation of unstructured texts at scale.
Our empirical study shows CoAnnotating to be an effective means to allocate work from results on different datasets, with up to 21% performance improvement over random baseline.
arXiv Detail & Related papers (2023-10-24T08:56:49Z) - OverPrompt: Enhancing ChatGPT through Efficient In-Context Learning [49.38867353135258]
We propose OverPrompt, leveraging the in-context learning capability of LLMs to handle multiple task inputs.
Our experiments show that OverPrompt can achieve cost-efficient zero-shot classification without causing significant detriment to task performance.
arXiv Detail & Related papers (2023-05-24T10:08:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.