Extracting Norms from Contracts Via ChatGPT: Opportunities and Challenges
- URL: http://arxiv.org/abs/2404.02269v1
- Date: Tue, 2 Apr 2024 19:49:34 GMT
- Title: Extracting Norms from Contracts Via ChatGPT: Opportunities and Challenges
- Authors: Amanul Haque, Munindar P. Singh,
- Abstract summary: We investigate the effectiveness of ChatGPT in extracting norms from contracts.
We find promising performance in norm extraction without requiring training or fine-tuning.
However, we find some limitations of ChatGPT in extracting these norms that lead to incorrect norm extractions.
- Score: 14.602364944958088
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We investigate the effectiveness of ChatGPT in extracting norms from contracts. Norms provide a natural way to engineer multiagent systems by capturing how to govern the interactions between two or more autonomous parties. We extract norms of commitment, prohibition, authorization, and power, along with associated norm elements (the parties involved, antecedents, and consequents) from contracts. Our investigation reveals ChatGPT's effectiveness and limitations in norm extraction from contracts. ChatGPT demonstrates promising performance in norm extraction without requiring training or fine-tuning, thus obviating the need for annotated data, which is not generally available in this domain. However, we found some limitations of ChatGPT in extracting these norms that lead to incorrect norm extractions. The limitations include oversight of crucial details, hallucination, incorrect parsing of conjunctions, and empty norm elements. Enhanced norm extraction from contracts can foster the development of more transparent and trustworthy formal agent interaction specifications, thereby contributing to the improvement of multiagent systems.
Related papers
- Normative Requirements Operationalization with Large Language Models [3.456725053685842]
Normative non-functional requirements specify constraints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms.
Recent research has tackled this challenge using a domain-specific language to specify normative requirements.
We propose a complementary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities.
arXiv Detail & Related papers (2024-04-18T17:01:34Z) - Towards Mitigating Perceived Unfairness in Contracts from a Non-Legal
Stakeholder's Perspective [2.9748898344267776]
We conduct an empirical study to analyze the perspectives of different stakeholders regarding contractual fairness.
We then investigate the ability of Pre-trained Language Models (PLMs) to identify unfairness in contractual sentences.
arXiv Detail & Related papers (2023-12-03T13:52:32Z) - Toward Unified Controllable Text Generation via Regular Expression
Instruction [56.68753672187368]
Our paper introduces Regular Expression Instruction (REI), which utilizes an instruction-based mechanism to fully exploit regular expressions' advantages to uniformly model diverse constraints.
Our method only requires fine-tuning on medium-scale language models or few-shot, in-context learning on large language models, and requires no further adjustment when applied to various constraint combinations.
arXiv Detail & Related papers (2023-09-19T09:05:14Z) - On Regularization and Inference with Label Constraints [62.60903248392479]
We compare two strategies for encoding label constraints in a machine learning pipeline, regularization with constraints and constrained inference.
For regularization, we show that it narrows the generalization gap by precluding models that are inconsistent with the constraints.
For constrained inference, we show that it reduces the population risk by correcting a model's violation, and hence turns the violation into an advantage.
arXiv Detail & Related papers (2023-07-08T03:39:22Z) - Pushing the Limits of ChatGPT on NLP Tasks [79.17291002710517]
Despite the success of ChatGPT, its performances on most NLP tasks are still well below the supervised baselines.
In this work, we looked into the causes, and discovered that its subpar performance was caused by the following factors.
We propose a collection of general modules to address these issues, in an attempt to push the limits of ChatGPT on NLP tasks.
arXiv Detail & Related papers (2023-06-16T09:40:05Z) - An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text
Generation [70.77243918587321]
Multi-aspect controllable text generation that controls generated text in multiple aspects has attracted increasing attention.
We provide a theoretical lower bound for the interference and empirically found that the interference grows with the number of layers where prefixes are inserted.
We propose using trainable gates to normalize the intervention of prefixes to restrain the growing interference.
arXiv Detail & Related papers (2022-12-19T11:53:59Z) - ContractNLI: A Dataset for Document-level Natural Language Inference for
Contracts [39.75232199445175]
We propose "document-level natural language inference (NLI) for contracts"
A system is given a set of hypotheses and a contract, and it is asked to classify whether each hypothesis is "entailed by", "contradicting to" or "not mentioned by" (neutral to) the contract.
We release the largest corpus to date consisting of 607 annotated contracts.
arXiv Detail & Related papers (2021-10-05T03:22:31Z) - Controllable Summarization with Constrained Markov Decision Process [50.04321779376415]
We study controllable text summarization which allows users to gain control on a particular attribute.
We propose a novel training framework based on Constrained Markov Decision Process (CMDP)
Our framework can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness.
arXiv Detail & Related papers (2021-08-07T09:12:53Z) - A Framework for Automatic Monitoring of Norms that regulate Time
Constrained Actions [0.0]
The proposed T-NORM model can be used to express abstract norms able to regulate classes of actions that should or should not be performed in a temporal interval.
We show how the model can be used to formalize obligations and prohibitions and for inhibiting them by introducing permissions and exemptions.
arXiv Detail & Related papers (2021-05-01T09:29:32Z) - Neural Contract Element Extraction Revisited: Letters from Sesame Street [13.389396129468745]
LSTM-based encoders perform better than dilated CNNs, Transformers, and BERT in this task.
domain-specific WORD2VEC embeddings outperform generic pre-trained GLOVE embeddings.
Morpho-syntactic features in the form of POS tag and token shape embeddings, as well as context-aware ELMO embeddings do not improve performance.
arXiv Detail & Related papers (2021-01-12T09:02:22Z) - Lexically-constrained Text Generation through Commonsense Knowledge
Extraction and Injection [62.071938098215085]
We focus on the Commongen benchmark, wherein the aim is to generate a plausible sentence for a given set of input concepts.
We propose strategies for enhancing the semantic correctness of the generated text.
arXiv Detail & Related papers (2020-12-19T23:23:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.