Parameter-Efficient Instruction Tuning of Large Language Models For Extreme Financial Numeral Labelling

• URL: http://arxiv.org/abs/2405.06671v2
• Date: Wed, 15 May 2024 14:43:23 GMT
• Title: Parameter-Efficient Instruction Tuning of Large Language Models For Extreme Financial Numeral Labelling
• Authors: Subhendu Khatuya, Rajdeep Mukherjee, Akash Ghosh, Manjunath Hegde, Koustuv Dasgupta, Niloy Ganguly, Saptarshi Ghosh, Pawan Goyal,
• Abstract summary: We study the problem of automatically annotating relevant numerals occurring in the financial documents with their corresponding tags. We propose a parameter efficient solution for the task using LoRA. Our proposed model, FLAN-FinXC, achieves new state-of-the-art performances on both the datasets.
• Score: 29.84946857859386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the problem of automatically annotating relevant numerals (GAAP metrics) occurring in the financial documents with their corresponding XBRL tags. Different from prior works, we investigate the feasibility of solving this extreme classification problem using a generative paradigm through instruction tuning of Large Language Models (LLMs). To this end, we leverage metric metadata information to frame our target outputs while proposing a parameter efficient solution for the task using LoRA. We perform experiments on two recently released financial numeric labeling datasets. Our proposed model, FLAN-FinXC, achieves new state-of-the-art performances on both the datasets, outperforming several strong baselines. We explain the better scores of our proposed model by demonstrating its capability for zero-shot as well as the least frequently occurring tags. Also, even when we fail to predict the XBRL tags correctly, our generated output has substantial overlap with the ground-truth in majority of the cases.

Related papers

• Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance [21.926934384262594]
  Large language models (LLMs) offer new opportunities to enhance the annotation process. We compare expert, crowd-sourced, and our LLM-based annotations in terms of agreement, label quality, and efficiency. Our findings reveal a substantial number of label errors, which, when corrected, induce a significant upward shift in reported model performance.
  arXiv  Detail & Related papers  (2024-10-24T16:27:03Z)
• LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Content [62.816876067499415]
  We propose LiveXiv: a scalable evolving live benchmark based on scientific ArXiv papers. LiveXiv accesses domain-specific manuscripts at any given timestamp and proposes to automatically generate visual question-answer pairs. We benchmark multiple open and proprietary Large Multi-modal Models (LMMs) on the first version of our benchmark, showing its challenging nature and exposing the models true abilities.
  arXiv  Detail & Related papers  (2024-10-14T17:51:23Z)
• Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels [75.77877889764073]
  Large Language Models (LLMs) have demonstrated remarkable performance through supervised fine-tuning or in-context learning using gold labels. This study explores whether solely utilizing unlabeled data can elicit strong model capabilities. We propose a new paradigm termed zero-to-strong generalization.
  arXiv  Detail & Related papers  (2024-09-19T02:59:44Z)
• SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
  Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts. We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM. We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
  arXiv  Detail & Related papers  (2024-07-16T04:41:58Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency [13.561104321425045]
  Large Language Models (LLMs) have demonstrated remarkable performance in data annotation tasks on general domain datasets. We investigate the potential of LLMs as efficient data annotators for extracting relations in financial documents. We demonstrate that the current state-of-the-art LLMs can be sufficient alternatives to non-expert crowdworkers.
  arXiv  Detail & Related papers  (2024-03-26T23:32:52Z)
• CELDA: Leveraging Black-box Language Model as Enhanced Classifier without Labels [14.285609493077965]
  Clustering-enhanced Linear Discriminative Analysis, a novel approach that improves the text classification accuracy with a very weak-supervision signal. Our framework draws a precise decision boundary without accessing weights or gradients of the LM model or data labels.
  arXiv  Detail & Related papers  (2023-06-05T08:35:31Z)
• Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization [34.85353544844499]
  We present the first dataset with fine-grained factual error annotations named DIASUMFACT. We define fine-grained factual error detection as a sentence-level multi-label classification problem. We propose an unsupervised model ENDERANKER via candidate ranking using pretrained encoder-decoder models.
  arXiv  Detail & Related papers  (2023-05-26T00:18:33Z)
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators [98.11286353828525]
  GPT-3.5 series models have demonstrated remarkable few-shot and zero-shot ability across various NLP tasks. We propose AnnoLLM, which adopts a two-step approach, explain-then-annotate. We build the first conversation-based information retrieval dataset employing AnnoLLM.
  arXiv  Detail & Related papers  (2023-03-29T17:03:21Z)
• Ground Truth Inference for Weakly Supervised Entity Matching [76.6732856489872]
  We propose a simple but powerful labeling model for weak supervision tasks. We then tailor the labeling model specifically to the task of entity matching. We show that our labeling model results in a 9% higher F1 score on average than the best existing method.
  arXiv  Detail & Related papers  (2022-11-13T17:57:07Z)
• Fortunately, Discourse Markers Can Enhance Language Models for Sentiment Analysis [13.149482582098429]
  We propose to leverage sentiment-carrying discourse markers to generate large-scale weakly-labeled data. We show the value of our approach on various benchmark datasets, including the finance domain.
  arXiv  Detail & Related papers  (2022-01-06T12:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.