Do Judges Behave Like Algorithms?

• URL: http://arxiv.org/abs/2608.10400v2
• Date: Wed, 12 Aug 2026 01:39:36 GMT
• Title: Do Judges Behave Like Algorithms?
• Abstract summary: We study judicial decision-making in misdemeanor bail hearings in Harris County, Texas.<n>Using available court data, we investigate whether magistrate judges follow what resembles an algorithm.<n>Our results reveal that these judges generally behave algorithmically.
• Score: 18.681366029551135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: What if judges already behave like algorithms? As artificial intelligence and algorithms are deployed in many settings, including the judicial system, many have debated whether judges should be allowed to rely on them. Instead, we ask whether judges follow predictable, algorithmic-like rules already. If judges already follow consistent, formula-like rules based on discrete and static factors such as criminal history, age, and charge type, then judicial behavior may be improved. However, if judges rely on individualized information that cannot be identified through court data, then standards-based decision-making may be more challenging to understand or improve. This work explores these questions by studying judicial decision-making in misdemeanor bail hearings in Harris County, Texas. Using available court data, we investigate whether magistrate judges follow what resembles an algorithm; whether they consider the same variables in their decision-making; and whether they are consistent with themselves and with each other. To do this, we train machine learning models for each judge, measure variable importance metrics to determine important variables for each judge's decision-making, and analyze outcomes of similar cases for judges. Our results reveal that these judges generally behave algorithmically: their decisions can be captured by small, interpretable formulas. However, in some cases, judges differ substantially, leading to surprising inconsistency and unequal treatment across similar defendants. Identifying cases where algorithms do not explain judicial decision-making can improve the justice system by focusing attention on decisions where individualized standards, rather than rules, better explains outcomes.

Related papers

• CyberJurors: A Multi-Agent Simulation Task for E-Commerce Disputes Verdict [59.06704011432714]
  E-commerce platforms have begun recruiting crowdsourced jurors to adjudicate massive volumes of transaction disputes.<n>We introduce a pioneering task, E-commerce Dispute Verdicts (EDV), and present VerdictBench, a multimodal benchmark comprising 6,000 real-world cases.<n>We propose CyberJurors, a multi-agent framework to clarify the dispute logic and regulate the verdict process.
  arXiv  Detail & Related papers  (2026-05-27T12:07:44Z)
• The Silicon Reasonable Person: Can AI Predict How Ordinary People Judge Reasonableness? [0.0]
  This Article investigates whether large language models (LLMs) can learn to identify patterns driving human reasonableness judgments.<n>We show that certain models capture not just surface-level responses but potentially their underlying decisional architecture.<n>These findings suggest practical applications: judges could calibrate intuitions against broader patterns, lawmakers could test policy interpretations, and resource-constrained litigants could preview argument reception.
  arXiv  Detail & Related papers  (2025-08-04T06:19:45Z)
• JudgeBench: A Benchmark for Evaluating LLM-based Judges [61.048125269475854]
  JudgeBench is a benchmark for evaluating LLM-based judges on challenging response pairs spanning knowledge, reasoning, math, and coding.<n>Our comprehensive evaluation on a collection of prompted judges, fine-tuned judges, multi-agent judges, and reward models shows that JudgeBench poses a significantly greater challenge than previous benchmarks.
  arXiv  Detail & Related papers  (2024-10-16T17:58:19Z)
• The Ethics of Automating Legal Actors [58.81546227716182]
  We argue that automating the role of the judge raises difficult ethical challenges, in particular for common law legal systems. Our argument follows from the social role of the judge in actively shaping the law, rather than merely applying it. Even in the case the models could achieve human-level capabilities, there would still be remaining ethical concerns inherent in the automation of the legal process.
  arXiv  Detail & Related papers  (2023-12-01T13:48:46Z)
• JudgeLM: Fine-tuned Large Language Models are Scalable Judges [48.053949045598344]
  evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively.<n>We propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks.<n>We train JudgeLM at different scales from 7B, 13B, to 33B parameters, and conduct a systematic analysis of its capabilities and behaviors.
  arXiv  Detail & Related papers  (2023-10-26T17:48:58Z)
• MUSER: A Multi-View Similar Case Retrieval Dataset [65.36779942237357]
  Similar case retrieval (SCR) is a representative legal AI application that plays a pivotal role in promoting judicial fairness. Existing SCR datasets only focus on the fact description section when judging the similarity between cases. We present M, a similar case retrieval dataset based on multi-view similarity measurement and comprehensive legal element with sentence-level legal element annotations.
  arXiv  Detail & Related papers  (2023-10-24T08:17:11Z)
• Algorithms, Incentives, and Democracy [0.0]
  We show how optimal classification by an algorithm designer can affect the distribution of behavior in a population. We then look at the effect of democratizing the rewards and punishments, or stakes, to the algorithmic classification to consider how a society can potentially stem (or facilitate!) predatory classification.
  arXiv  Detail & Related papers  (2023-07-05T14:22:01Z)
• Algorithmic Learning Foundations for Common Law [5.961705913076256]
  This paper looks at a common law legal system as a learning algorithm, models specific features of legal proceedings, and asks whether this system learns efficiently. A particular feature of our model is explicitly viewing various aspects of court proceedings as learning algorithms.
  arXiv  Detail & Related papers  (2022-09-07T00:39:36Z)
• Uncertainty in Criminal Justice Algorithms: simulation studies of the Pennsylvania Additive Classification Tool [0.0]
  We study the Pennsylvania Additive Classification Tool (PACT) that assigns custody levels to incarcerated individuals. We analyze the PACT in ways that criminal justice algorithms are often analyzed. We propose and carry out some new ways to study such algorithms.
  arXiv  Detail & Related papers  (2021-12-01T06:27:24Z)
• Equality before the Law: Legal Judgment Consistency Analysis for Fairness [55.91612739713396]
  In this paper, we propose an evaluation metric for judgment inconsistency, Legal Inconsistency Coefficient (LInCo) We simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups. We employ LInCo to explore the inconsistency in real cases and come to the following observations: (1) Both regional and gender inconsistency exist in the legal system, but gender inconsistency is much less than regional inconsistency.
  arXiv  Detail & Related papers  (2021-03-25T14:28:00Z)
• Predicting Court Decisions for Alimony: Avoiding Extra-legal Factors in Decision made by Judges and Not Understandable AI Models [0.02578242050187029]
  We present an explainable AI model designed in this purpose by combining a classification with random forest and a regression model. By using a large amount of court decisions in matters of divorce produced by French jurisdictions, we seek to identify if there may be extra-legal factors in the decisions taken by the judges.
  arXiv  Detail & Related papers  (2020-07-09T14:14:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.