From Text to Structure: Using Large Language Models to Support the
Development of Legal Expert Systems
- URL: http://arxiv.org/abs/2311.04911v1
- Date: Wed, 1 Nov 2023 18:31:02 GMT
- Title: From Text to Structure: Using Large Language Models to Support the
Development of Legal Expert Systems
- Authors: Samyar Janatian, Hannes Westermann, Jinzhe Tan, Jaromir Savelka, Karim
Benyekhlef
- Abstract summary: Rule-based expert systems focused on legislation can support laypeople in understanding how legislation applies to them and provide them with helpful context and information.
Here, we investigate what degree large language models (LLMs), such as GPT-4, are able to automatically extract structured representations from legislation.
We use LLMs to create pathways from legislation, according to the JusticeBot methodology for legal decision support systems, evaluate the pathways and compare them to manually created pathways.
- Score: 0.6249768559720122
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Encoding legislative text in a formal representation is an important
prerequisite to different tasks in the field of AI & Law. For example,
rule-based expert systems focused on legislation can support laypeople in
understanding how legislation applies to them and provide them with helpful
context and information. However, the process of analyzing legislation and
other sources to encode it in the desired formal representation can be
time-consuming and represents a bottleneck in the development of such systems.
Here, we investigate to what degree large language models (LLMs), such as
GPT-4, are able to automatically extract structured representations from
legislation. We use LLMs to create pathways from legislation, according to the
JusticeBot methodology for legal decision support systems, evaluate the
pathways and compare them to manually created pathways. The results are
promising, with 60% of generated pathways being rated as equivalent or better
than manually created ones in a blind comparison. The approach suggests a
promising path to leverage the capabilities of LLMs to ease the costly
development of systems based on symbolic approaches that are transparent and
explainable.
Related papers
- Large Language Models are Interpretable Learners [53.56735770834617]
In this paper, we show a combination of Large Language Models (LLMs) and symbolic programs can bridge the gap between expressiveness and interpretability.
The pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts.
As the knowledge learned by LSP is a combination of natural language descriptions and symbolic rules, it is easily transferable to humans (interpretable) and other LLMs.
arXiv Detail & Related papers (2024-06-25T02:18:15Z) - DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
We introduce DELTA, a discriminative model designed for legal case retrieval.
We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability.
Our approach can outperform existing state-of-the-art methods in legal case retrieval.
arXiv Detail & Related papers (2024-03-27T10:40:14Z) - Generative Context-aware Fine-tuning of Self-supervised Speech Models [54.389711404209415]
We study the use of generative large language models (LLM) generated context information.
We propose an approach to distill the generated information during fine-tuning of self-supervised speech models.
We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis.
arXiv Detail & Related papers (2023-12-15T15:46:02Z) - A Deep Learning-Based System for Automatic Case Summarization [2.9141777969894966]
This paper presents a deep learning-based system for efficient automatic case summarization.
The system offers both supervised and unsupervised methods to generate concise and relevant summaries of lengthy legal case documents.
Future work will focus on refining summarization techniques and exploring the application of our methods to other types of legal texts.
arXiv Detail & Related papers (2023-12-13T01:18:10Z) - Large Language Models and Explainable Law: a Hybrid Methodology [44.99833362998488]
The paper advocates for LLMs to enhance the accessibility, usage and explainability of rule-based legal systems.
A methodology is developed to explore the potential use of LLMs for translating the explanations produced by rule-based systems.
arXiv Detail & Related papers (2023-11-20T14:47:20Z) - Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model
Collaboration [52.57055162778548]
Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI.
Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems.
Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
arXiv Detail & Related papers (2023-10-13T16:47:20Z) - Large Language Models as Tax Attorneys: A Case Study in Legal
Capabilities Emergence [5.07013500385659]
This paper explores Large Language Models' (LLMs) capabilities in applying tax law.
Our experiments demonstrate emerging legal understanding capabilities, with improved performance in each subsequent OpenAI model release.
Findings indicate that LLMs, particularly when combined with prompting enhancements and the correct legal texts, can perform at high levels of accuracy but not yet at expert tax lawyer levels.
arXiv Detail & Related papers (2023-06-12T12:40:48Z) - Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural
Networks [3.5880535198436156]
We propose a novel graph-augmented dense statute retriever (G-DSR) model that incorporates the structure of legislation via a graph neural network to improve dense retrieval performance.
Experimental results show that our approach outperforms strong retrieval baselines on a real-world expert-annotated SAR dataset.
arXiv Detail & Related papers (2023-01-30T12:59:09Z) - Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents [56.40163943394202]
We release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding.
We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering.
arXiv Detail & Related papers (2021-05-09T09:39:25Z) - A Dataset for Statutory Reasoning in Tax Law Entailment and Question
Answering [37.66486350122862]
This paper investigates the performance of natural language understanding approaches on statutory reasoning.
We introduce a dataset, together with a legal-domain text corpus.
We contrast this with a hand-constructed Prolog-based system, designed to fully solve the task.
arXiv Detail & Related papers (2020-05-11T16:54:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.