Generating Clarification Questions for Disambiguating Contracts

• URL: http://arxiv.org/abs/2403.08053v1
• Date: Tue, 12 Mar 2024 19:57:39 GMT
• Title: Generating Clarification Questions for Disambiguating Contracts
• Authors: Anmol Singhal, Chirag Jain, Preethu Rose Anish, Arkajyoti Chakraborty, Smita Ghaisas
• Abstract summary: We introduce a novel legal NLP task that involves generating clarification questions for contracts. These questions aim to identify contract ambiguities on a document level, thereby assisting non-legal stakeholders. Experiments conducted on contracts sourced from the publicly available CUAD dataset show that ConRAP can detect ambiguities with an F2 score of 0.87.
• Score: 3.672364005691543
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enterprises frequently enter into commercial contracts that can serve as vital sources of project-specific requirements. Contractual clauses are obligatory, and the requirements derived from contracts can detail the downstream implementation activities that non-legal stakeholders, including requirement analysts, engineers, and delivery personnel, need to conduct. However, comprehending contracts is cognitively demanding and error-prone for such stakeholders due to the extensive use of Legalese and the inherent complexity of contract language. Furthermore, contracts often contain ambiguously worded clauses to ensure comprehensive coverage. In contrast, non-legal stakeholders require a detailed and unambiguous comprehension of contractual clauses to craft actionable requirements. In this work, we introduce a novel legal NLP task that involves generating clarification questions for contracts. These questions aim to identify contract ambiguities on a document level, thereby assisting non-legal stakeholders in obtaining the necessary details for eliciting requirements. This task is challenged by three core issues: (1) data availability, (2) the length and unstructured nature of contracts, and (3) the complexity of legal text. To address these issues, we propose ConRAP, a retrieval-augmented prompting framework for generating clarification questions to disambiguate contractual text. Experiments conducted on contracts sourced from the publicly available CUAD dataset show that ConRAP with ChatGPT can detect ambiguities with an F2 score of 0.87. 70% of the generated clarification questions are deemed useful by human evaluators.

Related papers

• Few-shot Policy (de)composition in Conversational Question Answering [54.259440408606515]
  We propose a neuro-symbolic framework to detect policy compliance using large language models (LLMs) in a few-shot setting. We show that our approach soundly reasons about policy compliance conversations by extracting sub-questions to be answered, assigning truth values from contextual information, and explicitly producing a set of logic statements from the given policies. We apply this approach to the popular PCD and conversational machine reading benchmark, ShARC, and show competitive performance with no task-specific finetuning.
  arXiv  Detail & Related papers  (2025-01-20T08:40:15Z)
• RIRAG: Regulatory Information Retrieval and Answer Generation [51.998738311700095]
  We introduce a task of generating question-passages pairs, where questions are automatically created and paired with relevant regulatory passages. We create the ObliQA dataset, containing 27,869 questions derived from the collection of Abu Dhabi Global Markets (ADGM) financial regulation documents. We design a baseline Regulatory Information Retrieval and Answer Generation (RIRAG) system and evaluate it with RePASs, a novel evaluation metric.
  arXiv  Detail & Related papers  (2024-09-09T14:44:19Z)
• Towards Mitigating Perceived Unfairness in Contracts from a Non-Legal Stakeholder's Perspective [2.9748898344267776]
  We conduct an empirical study to analyze the perspectives of different stakeholders regarding contractual fairness. We then investigate the ability of Pre-trained Language Models (PLMs) to identify unfairness in contractual sentences.
  arXiv  Detail & Related papers  (2023-12-03T13:52:32Z)
• PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition [63.51569687229681]
  We argue for the need to recognize the textual entailment relation of each proposition in a sentence individually. We propose PropSegmEnt, a corpus of over 45K propositions annotated by expert human raters. Our dataset structure resembles the tasks of (1) segmenting sentences within a document to the set of propositions, and (2) classifying the entailment relation of each proposition with respect to a different yet topically-aligned document.
  arXiv  Detail & Related papers  (2022-12-21T04:03:33Z)
• What to Read in a Contract? Party-Specific Summarization of Legal Obligations, Entitlements, and Prohibitions [27.92767201633733]
  Reviewing key obligations, entitlements, and prohibitions in legal contracts can be a tedious task due to their length and domain-specificity. We propose a new task of party-specific extractive summarization for legal contracts to facilitate faster reviewing and improved comprehension of rights and duties.
  arXiv  Detail & Related papers  (2022-12-19T19:53:14Z)
• ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction [84.0634340572349]
  We study automatic Contract Clause Extraction (CCE) by modeling implicit relations in legal contracts. In this work, we first comprehensively analyze the complexity issues of contracts and distill out three implicit relations commonly found in contracts. We propose a novel framework ConReader to exploit the above three relations for better contract understanding and improving CCE.
  arXiv  Detail & Related papers  (2022-10-17T02:15:18Z)
• Detecting Logical Relation In Contract Clauses [94.85352502638081]
  We develop an approach to automate the extraction of logical relations between clauses in a contract. The resulting approach should help contract authors detecting potential logical conflicts between clauses.
  arXiv  Detail & Related papers  (2021-11-02T19:26:32Z)
• ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts [39.75232199445175]
  We propose "document-level natural language inference (NLI) for contracts" A system is given a set of hypotheses and a contract, and it is asked to classify whether each hypothesis is "entailed by", "contradicting to" or "not mentioned by" (neutral to) the contract. We release the largest corpus to date consisting of 607 annotated contracts.
  arXiv  Detail & Related papers  (2021-10-05T03:22:31Z)
• A Benchmark for Lease Contract Review [9.249443355045969]
  We tackle the problem of detecting two different types of elements that play an important role in a contract review. The latter are terms or sentences that indicate that there is some danger or other potentially problematic situation for one or more of the signing parties. We release a new benchmark dataset of 179 lease agreement documents that we have manually annotated with the entities and red flags they contain.
  arXiv  Detail & Related papers  (2020-10-20T15:50:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.