OBJECTION! Lawyer Agents Mitigate Guilty Bias in Legal Judgment Prediction

• URL: http://arxiv.org/abs/2609.02158v1
• Date: Wed, 02 Sep 2026 06:18:44 GMT
• Title: OBJECTION! Lawyer Agents Mitigate Guilty Bias in Legal Judgment Prediction
• Abstract summary: We introduce OBJECTION, an inference-time pipeline that integrates an Adversarial Lawyer Agent into each 3-step reasoning of offense, unlawfulness, and culpability.<n>Unlike generic critics, our agent actively challenges the model's presumptions of guilt by injecting legal defense arguments at each reasoning stage.<n>Test results show that OBJECTION drastically reduces the False Guilty Rate (FGR) from 82.93% (SOTA baseline) to 16.69%.
• Score: 17.160212964113214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Legal Judgment Prediction (LJP) models are typically trained on documents that describe facts from a prosecutorial perspective. Existing datasets further exhibit severe label imbalance toward guilty outcomes. Consequently, these models suffer from "Guilty Bias", blindly accepting the prosecution's narrative as objective truth. Previous studies employing three-step reasoning structures or training on synthetically generated innocence data improve overall accuracy, but they still fail to mitigate bias at inference time. In this paper, we introduce OBJECTION, an inference-time pipeline that integrates an Adversarial Lawyer Agent into each 3-step reasoning of offense, unlawfulness, and culpability. Unlike generic critics, our agent actively challenges the model's presumptions of guilt by injecting legal defense arguments at each reasoning stage. To thoroughly evaluate this, we present a new "Natural Innocent" dataset including 3.4k real-world cases, overcoming the limitations of synthetic innocence benchmarks. Test results show that OBJECTION drastically reduces the False Guilty Rate (FGR) from 82.93% (SOTA baseline) to 16.69%, proving its capability to perform substantive legal reasoning. This work denotes a key progress toward aligning Legal AI with the presumption of innocence.

Related papers

• Legal Reasoning Is Not Lawyering: Rethinking Legal Benchmarks for Pro Se Access to Justice [0.0]
  Legal AI benchmark research frequently invokes the assumption that large language models can improve access to justice.<n>We argue that current benchmarks are not equipped to support this assumption because they evaluate legal reasoning over inputs that have already been preprocessed by legal experts.<n>We conclude by calling for legal benchmarks that directly measure robustness under pro se-like inputs so that access-to-justice claims about legal AI can become empirically testable.
  arXiv  Detail & Related papers  (2026-06-16T14:19:04Z)
• The Cases LJP Never Sees: Prosecution Decision Prediction for More Complete Criminal Liability Assessment [20.24311695645454]
  Legal Judgment Prediction (LJP) has become a core benchmark for evaluating AI in the criminal legal domain.<n>We propose textbfProsecution Decision Prediction ( PDP), the first Legal AI task built around prosecutorial review.
  arXiv  Detail & Related papers  (2026-05-27T13:32:18Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• ClaimGen-CN: A Large-scale Chinese Dataset for Legal Claim Generation [56.79698529022327]
  Legal claims refer to the plaintiff's demands in a case and are essential to guiding judicial reasoning and case resolution.<n>This paper explores the problem of legal claim generation based on the given case's facts.<n>We construct ClaimGen-CN, the first dataset for Chinese legal claim generation task.
  arXiv  Detail & Related papers  (2025-08-24T07:19:25Z)
• AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction [56.797874973414636]
  AnnoCaseLaw is a first-of-its-kind dataset of 471 meticulously annotated U.S. Appeals Court negligence cases.<n>Our dataset lays the groundwork for more human-aligned, explainable Legal Judgment Prediction models.<n>Results demonstrate that LJP remains a formidable task, with application of legal precedent proving particularly difficult.
  arXiv  Detail & Related papers  (2025-02-28T19:14:48Z)
• Beyond Guilt: Legal Judgment Prediction with Trichotomous Reasoning [12.589047235741194]
  We introduce LJPIV, the first benchmark dataset for Legal Judgment Prediction with Innocent Verdicts.<n>Adhering to the trichotomous dogmatics, we extend three widely-used legal datasets through LLM-based augmentation and manual verification.<n>Our experiments with state-of-the-art legal LLMs and novel strategies that integrate trichotomous reasoning into zero-shot prompting and fine-tuning reveal: (1) current legal LLMs have significant room for improvement, with even the best models achieving an F1 score of less than 0.3 on LJPIV.
  arXiv  Detail & Related papers  (2024-12-19T07:14:13Z)
• Distinguish Confusion in Legal Judgment Prediction via Revised Relation Knowledge [38.58529647679356]
  Legal Judgment Prediction aims to automatically predict a law case's judgment results based on the text description of its facts. The confusing law articles (or charges) problem frequently occurs, reflecting that the law cases applicable to similar articles (or charges) tend to be misjudged. This paper proposes an end-to-end model named textitD-LADAN to solve the above challenges.
  arXiv  Detail & Related papers  (2024-08-18T09:44:59Z)
• Towards Explainability in Legal Outcome Prediction Models [64.00172507827499]
  We argue that precedent is a natural way of facilitating explainability for legal NLP models. By developing a taxonomy of legal precedent, we are able to compare human judges and neural models. We find that while the models learn to predict outcomes reasonably well, their use of precedent is unlike that of human judges.
  arXiv  Detail & Related papers  (2024-03-25T15:15:41Z)
• Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction [46.71918729837462]
  Given the fact description text of a legal case, legal judgment prediction aims to predict the case's charge, law article and penalty term. Previous studies fail to distinguish different classification errors with a standard cross-entropy classification loss. We propose a moco-based supervised contrastive learning to learn distinguishable representations. We further enhance the representation of the fact description with extracted crime amounts which are encoded by a pre-trained numeracy model.
  arXiv  Detail & Related papers  (2022-11-15T15:53:56Z)
• Do Charge Prediction Models Learn Legal Theory? [59.74220430434435]
  We argue that trustworthy charge prediction models should take legal theories into consideration. We propose three principles for trustworthy models should follow in this task, which are sensitive, selective, and presumption of innocence. Our findings indicate that, while existing charge prediction models meet the selective principle on a benchmark dataset, most of them are still not sensitive enough and do not satisfy the presumption of innocence.
  arXiv  Detail & Related papers  (2022-10-31T07:32:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.