Persuadability and LLMs as Legal Decision Tools

• URL: http://arxiv.org/abs/2604.26233v1
• Date: Wed, 29 Apr 2026 02:28:34 GMT
• Title: Persuadability and LLMs as Legal Decision Tools
• Abstract summary: Large Language Models (LLMs) are proposed as legal decision assistants, and even first-instance decision-makers.<n>We explore how frontier open- and closed-weights LLMs respond to legal arguments.<n>We report original experimental results examining how the quality of the advocate making those arguments affects the likelihood that a model will agree with a particular legal point of view.
• Score: 0.21485350418225244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Large Language Models (LLMs) are proposed as legal decision assistants, and even first-instance decision-makers, across a range of judicial and administrative contexts, it becomes essential to explore how they answer legal questions, and in particular the factors that lead them to decide difficult questions in one way or another. A specific feature of legal decisions is the need to respond to arguments advanced by contending parties. A legal decision-maker must be able to engage with, and respond to, including through being potentially persuaded by, arguments advanced by the parties. Conversely, they should not be unduly persuadable, influenced by a particularly compelling advocate to decide cases based on the skills of the advocates, rather than the merits of the case. We explore how frontier open- and closed-weights LLMs respond to legal arguments, reporting original experimental results examining how the quality of the advocate making those arguments affects the likelihood that a model will agree with a particular legal point of view, and exploring the factors driving these results. Our results have implications for the feasibility of adopting LLMs across legal and administrative settings.

Related papers

• WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report [55.27414605169639]
  We present WisdomInterrogatory (LuWen), an open-source Chinese legal language model built upon the Baichuan foundation model.<n>We evaluate LuWen on five representative legal tasks spanning both prediction and generation settings.
  arXiv  Detail & Related papers  (2026-04-08T06:59:07Z)
• LegalOne: A Family of Foundation Models for Reliable Legal Reasoning [54.57434222018289]
  We present LegalOne, a family of foundational models specifically tailored for the Chinese legal domain.<n>LegalOne is developed through a comprehensive three-phase pipeline designed to master legal reasoning.<n>We publicly release the LegalOne weights and the LegalKit evaluation framework to advance the field of Legal AI.
  arXiv  Detail & Related papers  (2026-01-31T10:18:32Z)
• PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice [67.71760070255425]
  We introduce PLawBench, a practical benchmark for evaluating large language models (LLMs) in legal practice scenarios.<n>PLawBench comprises 850 questions across 13 practical legal scenarios, with each question accompanied by expert-designed evaluation rubrics.<n>Using an LLM-based evaluator aligned with human expert judgments, we evaluate 10 state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2026-01-23T11:36:10Z)
• Argumentation-Based Explainability for Legal AI: Comparative and Regulatory Perspectives [0.9668407688201359]
  Artificial Intelligence (AI) systems are increasingly deployed in legal contexts.<n>The so-called black box problem'' undermines legitimacy of automated decision-making.<n>XAI has proposed a variety of methods to enhance transparency.
  arXiv  Detail & Related papers  (2025-10-13T07:19:15Z)
• Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarization [44.55119228567464]
  Large Language Models (LLMs) are increasingly used to generate user-tailored summaries.<n>This raises important questions about motivated reasoning.<n>We investigate how LLMs respond to prompts conditioned on different legal roles.
  arXiv  Detail & Related papers  (2025-08-30T15:22:47Z)
• Judicial Requirements for Generative AI in Legal Reasoning [0.0]
  Large Language Models (LLMs) are being integrated into professional domains, yet their limitations in high-stakes fields like law remain poorly understood.<n>This paper defines the core capabilities that an AI system must possess to function as a reliable reasoning tool in judicial decision-making.
  arXiv  Detail & Related papers  (2025-08-26T09:56:26Z)
• GLARE: Agentic Reasoning for Legal Judgment Prediction [60.13483016810707]
  Legal judgment prediction (LJP) has become increasingly important in the legal field.<n>Existing large language models (LLMs) have significant problems of insufficient reasoning due to a lack of legal knowledge.<n>We introduce GLARE, an agentic legal reasoning framework that dynamically acquires key legal knowledge by invoking different modules.
  arXiv  Detail & Related papers  (2025-08-22T13:38:12Z)
• NomicLaw: Emergent Trust and Strategic Argumentation in LLMs During Collaborative Law-Making [6.56837855642886]
  We introduce NomicLaw, a structured multi-agent simulation where LLMs engage in collaborative law-making.<n>We quantitatively measure trust and reciprocity via voting patterns and qualitatively assess how agents use strategic language to justify proposals.<n>Our results highlight the latent social reasoning and persuasive capabilities of ten open-source LLMs and provide insights into the design of future AI systems.
  arXiv  Detail & Related papers  (2025-08-07T12:49:44Z)
• Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession [0.0]
  This article reports the results of a study examining the ability of legal and nonlegal Large Language Models to perform legal analysis.<n>The results show that LLMs can conduct basic IRAC analysis, but are limited by brief responses lacking detail, an inability to commit to answers, false confidence, and hallucinations.
  arXiv  Detail & Related papers  (2025-02-04T19:50:48Z)
• Using LLMs to Discover Legal Factors [0.6249768559720122]
  We use large language models to discover factors that effectively represent a legal domain. Our method takes as input raw court opinions and produces a set of factors and associated definitions. We demonstrate that a semi-automated approach, incorporating minimal human involvement, produces factor representations that can predict case outcomes with moderate success.
  arXiv  Detail & Related papers  (2024-10-10T00:42:10Z)
• Argumentative Large Language Models for Explainable and Contestable Claim Verification [13.045050015831903]
  We introduce ArgLLMs, a method for augmenting large language models with argumentative reasoning.<n>ArgLLMs construct argumentation frameworks, which then serve as the basis for formal reasoning in support of decision-making.<n>We evaluate ArgLLMs' performance experimentally in comparison with state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-05-03T13:12:28Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.