Large Language Model Prompt Chaining for Long Legal Document Classification

• URL: http://arxiv.org/abs/2308.04138v1
• Date: Tue, 8 Aug 2023 08:57:01 GMT
• Title: Large Language Model Prompt Chaining for Long Legal Document Classification
• Authors: Dietrich Trautmann
• Abstract summary: Chaining is a strategy used to decompose complex tasks into smaller, manageable components. We demonstrate that through prompt chaining, we can not only enhance the performance over zero-shot, but also surpass the micro-F1 score achieved by larger models.
• Score: 2.3148470932285665
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompting is used to guide or steer a language model in generating an appropriate response that is consistent with the desired outcome. Chaining is a strategy used to decompose complex tasks into smaller, manageable components. In this study, we utilize prompt chaining for extensive legal document classification tasks, which present difficulties due to their intricate domain-specific language and considerable length. Our approach begins with the creation of a concise summary of the original document, followed by a semantic search for related exemplar texts and their corresponding annotations from a training corpus. Finally, we prompt for a label - based on the task - to assign, by leveraging the in-context learning from the few-shot prompt. We demonstrate that through prompt chaining, we can not only enhance the performance over zero-shot, but also surpass the micro-F1 score achieved by larger models, such as ChatGPT zero-shot, using smaller models.

Related papers

• Graph-DPEP: Decomposed Plug and Ensemble Play for Few-Shot Document Relation Extraction with Graph-of-Thoughts Reasoning [34.85741925091139]
  Graph-DPEP framework is grounded in the reasoning behind triplet explanation thoughts presented in natural language. We develop "ensemble-play", reapplying generation on the entire type list by leveraging the reasoning thoughts embedded in a sub-graph.
  arXiv  Detail & Related papers  (2024-11-05T07:12:36Z)
• Manual Verbalizer Enrichment for Few-Shot Text Classification [1.860409237919611]
  acrshortmave is an approach for verbalizer construction by enrichment of class labels. Our model achieves state-of-the-art results while using significantly fewer resources.
  arXiv  Detail & Related papers  (2024-10-08T16:16:47Z)
• Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA [51.3033125256716]
  We model the subgraph retrieval task as a conditional generation task handled by small language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, competitive retrieval performance compared to state-of-the-art models. Our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks.
  arXiv  Detail & Related papers  (2024-10-08T15:22:36Z)
• Dense X Retrieval: What Retrieval Granularity Should We Use? [56.90827473115201]
  Often-overlooked design choice is the retrieval unit in which the corpus is indexed, e.g. document, passage, or sentence. We introduce a novel retrieval unit, proposition, for dense retrieval. Experiments reveal that indexing a corpus by fine-grained units such as propositions significantly outperforms passage-level units in retrieval tasks.
  arXiv  Detail & Related papers  (2023-12-11T18:57:35Z)
• OLaLa: Ontology Matching with Large Language Models [2.211868306499727]
  Ontology Matching is a challenging task where information in natural language is one of the most important signals to process. With the rise of Large Language Models, it is possible to incorporate this knowledge in a better way into the matching pipeline. We show that with only a handful of examples and a well-designed prompt, it is possible to achieve results that are en par with supervised matching systems.
  arXiv  Detail & Related papers  (2023-11-07T09:34:20Z)
• MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text Classification [65.51149771074944]
  MetricPrompt eases verbalizer design difficulty by reformulating few-shot text classification task into text pair relevance estimation task. We conduct experiments on three widely used text classification datasets across four few-shot settings. Results show that MetricPrompt outperforms manual verbalizer and other automatic verbalizer design methods across all few-shot settings.
  arXiv  Detail & Related papers  (2023-06-15T06:51:35Z)
• Automated Few-shot Classification with Instruction-Finetuned Language Models [76.69064714392165]
  We show that AuT-Few outperforms state-of-the-art few-shot learning methods. We also show that AuT-Few is the best ranking method across datasets on the RAFT few-shot benchmark.
  arXiv  Detail & Related papers  (2023-05-21T21:50:27Z)
• LabelPrompt: Effective Prompt-based Learning for Relation Classification [31.291466190218912]
  This paper presents a novel prompt-based learning method, namely LabelPrompt, for the relation classification task. Motivated by the intuition to GIVE MODEL CHOICES!'', we first define additional tokens to represent relation labels, which regard these tokens as the verbaliser with semantic initialisation. Then, to mitigate inconsistency between predicted relations and given entities, we implement an entity-aware module with contrastive learning.
  arXiv  Detail & Related papers  (2023-02-16T04:06:25Z)
• Generative Prompt Tuning for Relation Classification [21.027631157115135]
  We propose a novel generative prompt tuning method to reformulate relation classification as an infilling problem. In addition, we design entity-guided decoding and discriminative relation scoring to generate and align relations effectively and efficiently during inference.
  arXiv  Detail & Related papers  (2022-10-22T12:40:23Z)
• Text Summarization with Latent Queries [60.468323530248945]
  We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms. Under a deep generative framework, our system jointly optimize a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time. Our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.
  arXiv  Detail & Related papers  (2021-05-31T21:14:58Z)
• Pre-training via Paraphrasing [96.79972492585112]
  We introduce MARGE, a pre-trained sequence-to-sequence model learned with an unsupervised multi-lingual paraphrasing objective. We show it is possible to jointly learn to do retrieval and reconstruction, given only a random initialization. For example, with no additional task-specific training we achieve BLEU scores of up to 35.8 for document translation.
  arXiv  Detail & Related papers  (2020-06-26T14:43:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.