When Does Defendant Statement Matter? A Study of Bias and Persuasion in LLM-Simulated Jurors

• URL: http://arxiv.org/abs/2609.09887v1
• Date: Wed, 09 Sep 2026 08:43:34 GMT
• Title: When Does Defendant Statement Matter? A Study of Bias and Persuasion in LLM-Simulated Jurors
• Abstract summary: We study when and how a defendant's courtroom statement affects LLM-simulated jurors.<n>JuryBench is a benchmark containing controversial criminal cases in U.S. criminal law.<n>We find that emotional persuasion can be detrimental, since jurors may perceive it as evidence of guilt or inconsistency.<n>Background fit between jurors and defendants is a stronger and significant factor than other isolated factors.
• Score: 2.8935588665357077
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: LLMs have been used to simulate human decision-making in professional settings, yet their behaviors in common-law jury trials remain unexplored. We study when and how a defendant's courtroom statement affects LLM-simulated jurors, focusing on persuasion, ideological bias, and background-based affinity. To support the analysis, we introduce JuryBench, a benchmark containing controversial criminal cases in U.S. criminal law. In each case, a defendant can claim various plausible justifications to support acquittal or reduced liability. We fix the base case and design defendants of different backgrounds, who give courtroom statements with varying emotional appeal or rebuttal. Jurors with diverse ideological profiles across the spectrum are simulated. We examine 20 frontier LLMs, resulting in a total of 432K decisions and rationales, and quantify changes in verdict severity. Our findings show that LLM-jury simulation echoes many human-jury findings. First, emotional persuasion can be detrimental, since jurors may perceive it as evidence of guilt or inconsistency. Next, we show that background fit between jurors and defendants is a stronger and significant factor than other isolated factors, and that jurors are in general harsher toward opposite-background defendants and lenient toward same-background ones. Finally, we find that juror ideology also strongly shapes severity judgments. These findings highlight both the promise and risks of using LLMs to model jury reasoning and call for careful evaluation. The data and code are available at https://github.com/choyingw/JuryBench

Related papers

• Reasoning Jury: Multi-Model Consensus for Evaluating Reasoning Traces [8.727904788574602]
  Reasoning Jury is a system that replaces the single judge with a jury of LLMs.<n>In Reasoning Jury, defects of a reasoning trace and their severity are surfaced through a deliberation.<n>We show that Reasoning Jury with a jury of open-weight models is able to significantly outperform frontier models.
  arXiv  Detail & Related papers  (2026-08-12T21:00:22Z)
• CyberJurors: A Multi-Agent Simulation Task for E-Commerce Disputes Verdict [59.06704011432714]
  E-commerce platforms have begun recruiting crowdsourced jurors to adjudicate massive volumes of transaction disputes.<n>We introduce a pioneering task, E-commerce Dispute Verdicts (EDV), and present VerdictBench, a multimodal benchmark comprising 6,000 real-world cases.<n>We propose CyberJurors, a multi-agent framework to clarify the dispute logic and regulate the verdict process.
  arXiv  Detail & Related papers  (2026-05-27T12:07:44Z)
• How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality [9.19183567561999]
  Large language models (LLMs) has been widely adopted as a scalable surrogate for human evaluation, yet such judges remain imperfect and susceptible to surface-level biases.<n>With the rise of reasoning-capable models, exposing a generator's reasoning content to the judge provides richer information and is a natural candidate for improving judgment accuracy.<n>We find that weak judges are easily swayed by reasoning presence, while strong judges can partially leverage reasoning as informative evidence.
  arXiv  Detail & Related papers  (2026-04-08T07:21:18Z)
• Profiling vs. Case-specific Evidence: A Probabilistic Analysis [0.0]
  We argue that profiling evidence may support a generic hypothesis, but is not evidence that the defendant is guilty of the specific crime which they are accused of.<n>Our critique departs from others by grounding the argument in a probabilistic analysis of evidentiary value.
  arXiv  Detail & Related papers  (2026-02-17T13:43:02Z)
• Quantitative LLM Judges [60.773734899532336]
  We propose quantitative LLM judges, which align evaluation scores of existing LLM judges to humans in a given domain.<n>The models are trained to improve the score of the original judge using its rationale and score.<n>Our experiments show that quantitative judges can improve the predictive power of existing judges through post-hoc modeling.
  arXiv  Detail & Related papers  (2025-06-03T14:44:23Z)
• Arbiters of Ambivalence: Challenges of Using LLMs in No-Consensus Tasks [52.098988739649705]
  This study examines the biases and limitations of LLMs in three roles: answer generator, judge, and debater.<n>We develop a no-consensus'' benchmark by curating examples that encompass a variety of a priori ambivalent scenarios.<n>Our results show that while LLMs can provide nuanced assessments when generating open-ended answers, they tend to take a stance on no-consensus topics when employed as judges or debaters.
  arXiv  Detail & Related papers  (2025-05-28T01:31:54Z)
• JudgeBench: A Benchmark for Evaluating LLM-based Judges [61.048125269475854]
  JudgeBench is a benchmark for evaluating LLM-based judges on challenging response pairs spanning knowledge, reasoning, math, and coding.<n>Our comprehensive evaluation on a collection of prompted judges, fine-tuned judges, multi-agent judges, and reward models shows that JudgeBench poses a significantly greater challenge than previous benchmarks.
  arXiv  Detail & Related papers  (2024-10-16T17:58:19Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks [49.60689355674541]
  A rich literature in cognitive science has studied people's causal and moral intuitions. This work has revealed a number of factors that systematically influence people's judgments. We test whether large language models (LLMs) make causal and moral judgments about text-based scenarios that align with human participants.
  arXiv  Detail & Related papers  (2023-10-30T15:57:32Z)
• JudgeLM: Fine-tuned Large Language Models are Scalable Judges [48.053949045598344]
  evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively.<n>We propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks.<n>We train JudgeLM at different scales from 7B, 13B, to 33B parameters, and conduct a systematic analysis of its capabilities and behaviors.
  arXiv  Detail & Related papers  (2023-10-26T17:48:58Z)
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena [76.21004582932268]
  We examine the usage and limitations of LLM-as-a-judge, including position, verbosity, and self-enhancement biases. We then verify the agreement between LLM judges and human preferences by introducing two benchmarks: MT-bench, a multi-turn question set; and Arena, a crowdsourced battle platform.
  arXiv  Detail & Related papers  (2023-06-09T05:55:52Z)
• AI-lead Court Debate Case Investigation [24.221021061471177]
  In the judicial trial, it can help the judge raise efficient questions so that the judge has a clearer understanding of the case. In this work, we propose an innovative end-to-end question generation model-Trial Brain Model (TBM) to build a Trial Brain.
  arXiv  Detail & Related papers  (2020-10-22T11:05:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.