Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance

• URL: http://arxiv.org/abs/2308.14634v1
• Date: Mon, 28 Aug 2023 15:04:16 GMT
• Title: Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance
• Authors: Lefteris Loukas, Ilias Stogiannidis, Prodromos Malakasiotis, Stavros Vassos
• Abstract summary: In-context learning with GPT-3.5 and GPT-4 minimizes the technical expertise required and eliminates the need for expensive GPU computing. We fine-tune other pre-trained, masked language models with SetFit to achieve state-of-the-art results both in full-data and few-shot settings. Our findings show that querying GPT-3.5 and GPT-4 can outperform fine-tuned, non-generative models even with fewer examples.
• Score: 4.305568120980929
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose the use of conversational GPT models for easy and quick few-shot text classification in the financial domain using the Banking77 dataset. Our approach involves in-context learning with GPT-3.5 and GPT-4, which minimizes the technical expertise required and eliminates the need for expensive GPU computing while yielding quick and accurate results. Additionally, we fine-tune other pre-trained, masked language models with SetFit, a recent contrastive learning technique, to achieve state-of-the-art results both in full-data and few-shot settings. Our findings show that querying GPT-3.5 and GPT-4 can outperform fine-tuned, non-generative models even with fewer examples. However, subscription fees associated with these solutions may be considered costly for small organizations. Lastly, we find that generative models perform better on the given task when shown representative samples selected by a human expert rather than when shown random ones. We conclude that a) our proposed methods offer a practical solution for few-shot tasks in datasets with limited label availability, and b) our state-of-the-art results can inspire future work in the area.

Related papers

• A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classification [0.8192907805418583]
  Large Language Models (LLMs) have demonstrated impressive capabilities across diverse Natural Language Processing (NLP) tasks. This study investigates the efficacy of instruction fine-tuning to enhance their performance in financial text classification tasks.
  arXiv  Detail & Related papers  (2024-11-04T18:06:36Z)
• Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning [0.0]
  Non-generative, small-sized models can outperform GPT-3.5 and GPT-4 models in zero-shot learning settings. Fine-tuned models show comparable results to GPT-3.5 when it is fine-tuned on the task of determining market sentiment.
  arXiv  Detail & Related papers  (2024-08-22T09:10:43Z)
• One-Shot Open Affordance Learning with Foundation Models [54.15857111929812]
  We introduce One-shot Open Affordance Learning (OOAL), where a model is trained with just one example per base object category. We propose a vision-language framework with simple and effective designs that boost the alignment between visual features and affordance text embeddings. Experiments on two affordance segmentation benchmarks show that the proposed method outperforms state-of-the-art models with less than 1% of the full training data.
  arXiv  Detail & Related papers  (2023-11-29T16:23:06Z)
• Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking [3.9412826185755017]
  Few-shot and large language models (LLMs) can perform effectively with just 1-5 examples per class. Our work addresses the performance-cost trade-offs of these methods over the Banking77 financial intent detection dataset. To inspire future research, we provide a human expert's curated subset of Banking77, along with extensive error analysis.
  arXiv  Detail & Related papers  (2023-11-10T15:10:36Z)
• Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks [36.84636748560657]
  Large language models such as ChatGPT and GPT-4 have shown exceptional capabilities of generalist models. How effective are such models in the financial domain?
  arXiv  Detail & Related papers  (2023-05-10T03:13:54Z)
• Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification [59.698811329287174]
  We leverage GPT-2 for generating artificial training instances in order to improve classification performance. Our results show that fine-tuning GPT-2 in a handful of label instances leads to consistent classification improvements.
  arXiv  Detail & Related papers  (2021-11-17T12:10:03Z)
• Reframing Instructional Prompts to GPTk's Language [72.69833640335519]
  We propose reframing techniques for model designers to create effective prompts for language models. Our results show that reframing improves few-shot learning performance by 14% while reducing sample complexity. The performance gains are particularly important on large language models, such as GPT3 where tuning models or prompts on large datasets is not feasible.
  arXiv  Detail & Related papers  (2021-09-16T09:44:43Z)
• What Makes Good In-Context Examples for GPT-$3$? [101.99751777056314]
  GPT-$3$ has attracted lots of attention due to its superior performance across a wide range of NLP tasks. Despite its success, we found that the empirical results of GPT-$3$ depend heavily on the choice of in-context examples. In this work, we investigate whether there are more effective strategies for judiciously selecting in-context examples.
  arXiv  Detail & Related papers  (2021-01-17T23:38:40Z)
• Evaluating data augmentation for financial time series classification [85.38479579398525]
  We evaluate several augmentation methods applied to stocks datasets using two state-of-the-art deep learning models. For a relatively small dataset augmentation methods achieve up to $400%$ improvement in risk adjusted return performance. For a larger stock dataset augmentation methods achieve up to $40%$ improvement.
  arXiv  Detail & Related papers  (2020-10-28T17:53:57Z)
• Investigating Pretrained Language Models for Graph-to-Text Generation [55.55151069694146]
  Graph-to-text generation aims to generate fluent texts from graph-based data. We present a study across three graph domains: meaning representations, Wikipedia knowledge graphs (KGs) and scientific KGs. We show that the PLMs BART and T5 achieve new state-of-the-art results and that task-adaptive pretraining strategies improve their performance even further.
  arXiv  Detail & Related papers  (2020-07-16T16:05:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.