Large Language Model Prompt Chaining for Long Legal Document
Classification
- URL: http://arxiv.org/abs/2308.04138v1
- Date: Tue, 8 Aug 2023 08:57:01 GMT
- Title: Large Language Model Prompt Chaining for Long Legal Document
Classification
- Authors: Dietrich Trautmann
- Abstract summary: Chaining is a strategy used to decompose complex tasks into smaller, manageable components.
We demonstrate that through prompt chaining, we can not only enhance the performance over zero-shot, but also surpass the micro-F1 score achieved by larger models.
- Score: 2.3148470932285665
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Prompting is used to guide or steer a language model in generating an
appropriate response that is consistent with the desired outcome. Chaining is a
strategy used to decompose complex tasks into smaller, manageable components.
In this study, we utilize prompt chaining for extensive legal document
classification tasks, which present difficulties due to their intricate
domain-specific language and considerable length. Our approach begins with the
creation of a concise summary of the original document, followed by a semantic
search for related exemplar texts and their corresponding annotations from a
training corpus. Finally, we prompt for a label - based on the task - to
assign, by leveraging the in-context learning from the few-shot prompt. We
demonstrate that through prompt chaining, we can not only enhance the
performance over zero-shot, but also surpass the micro-F1 score achieved by
larger models, such as ChatGPT zero-shot, using smaller models.
Related papers
- Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness [3.2925222641796554]
"pointer-guided segment ordering" (SO) is a novel pre-training technique aimed at enhancing the contextual understanding of paragraph-level text representations.
Our experiments show that pointer-guided pre-training significantly enhances the model's ability to understand complex document structures.
arXiv Detail & Related papers (2024-06-06T15:17:51Z) - FastGAS: Fast Graph-based Annotation Selection for In-Context Learning [53.17606395275021]
In-context learning (ICL) empowers large language models (LLMs) to tackle new tasks by using a series of training instances as prompts.
Existing methods have proposed to select a subset of unlabeled examples for annotation.
We propose a graph-based selection method, FastGAS, designed to efficiently identify high-quality instances.
arXiv Detail & Related papers (2024-06-06T04:05:54Z) - A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervision [74.972172804514]
We introduce a multi-task Transformer model, CSLR2, that is able to ingest a signing sequence and output in a joint embedding space between signed language and spoken language text.
New dataset annotations provide continuous sign-level annotations for six hours of test videos, and will be made publicly available.
Our model significantly outperforms the previous state of the art on both tasks.
arXiv Detail & Related papers (2024-05-16T17:19:06Z) - Improving Long Text Understanding with Knowledge Distilled from Summarization Model [17.39913210351487]
We propose our emphGist Detector to leverage the gist detection ability of a summarization model.
Gist Detector first learns the gist detection knowledge distilled from a summarization model, and then produces gist-aware representations.
We evaluate our method on three different tasks: long document classification, distantly supervised open-domain question answering, and non-parallel text style transfer.
arXiv Detail & Related papers (2024-05-08T10:49:39Z) - OLaLa: Ontology Matching with Large Language Models [2.211868306499727]
Ontology Matching is a challenging task where information in natural language is one of the most important signals to process.
With the rise of Large Language Models, it is possible to incorporate this knowledge in a better way into the matching pipeline.
We show that with only a handful of examples and a well-designed prompt, it is possible to achieve results that are en par with supervised matching systems.
arXiv Detail & Related papers (2023-11-07T09:34:20Z) - MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text
Classification [65.51149771074944]
MetricPrompt eases verbalizer design difficulty by reformulating few-shot text classification task into text pair relevance estimation task.
We conduct experiments on three widely used text classification datasets across four few-shot settings.
Results show that MetricPrompt outperforms manual verbalizer and other automatic verbalizer design methods across all few-shot settings.
arXiv Detail & Related papers (2023-06-15T06:51:35Z) - Automated Few-shot Classification with Instruction-Finetuned Language
Models [76.69064714392165]
We show that AuT-Few outperforms state-of-the-art few-shot learning methods.
We also show that AuT-Few is the best ranking method across datasets on the RAFT few-shot benchmark.
arXiv Detail & Related papers (2023-05-21T21:50:27Z) - LabelPrompt: Effective Prompt-based Learning for Relation Classification [31.291466190218912]
This paper presents a novel prompt-based learning method, namely LabelPrompt, for the relation classification task.
Motivated by the intuition to GIVE MODEL CHOICES!'', we first define additional tokens to represent relation labels, which regard these tokens as the verbaliser with semantic initialisation.
Then, to mitigate inconsistency between predicted relations and given entities, we implement an entity-aware module with contrastive learning.
arXiv Detail & Related papers (2023-02-16T04:06:25Z) - Generative Prompt Tuning for Relation Classification [21.027631157115135]
We propose a novel generative prompt tuning method to reformulate relation classification as an infilling problem.
In addition, we design entity-guided decoding and discriminative relation scoring to generate and align relations effectively and efficiently during inference.
arXiv Detail & Related papers (2022-10-22T12:40:23Z) - Text Summarization with Latent Queries [60.468323530248945]
We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms.
Under a deep generative framework, our system jointly optimize a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time.
Our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.
arXiv Detail & Related papers (2021-05-31T21:14:58Z) - Pre-training via Paraphrasing [96.79972492585112]
We introduce MARGE, a pre-trained sequence-to-sequence model learned with an unsupervised multi-lingual paraphrasing objective.
We show it is possible to jointly learn to do retrieval and reconstruction, given only a random initialization.
For example, with no additional task-specific training we achieve BLEU scores of up to 35.8 for document translation.
arXiv Detail & Related papers (2020-06-26T14:43:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.