Related papers: Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines

Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines

URL: http://arxiv.org/abs/2607.05680v1
Date: Mon, 06 Jul 2026 22:50:14 GMT
Title: Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines
Abstract summary: We study how people evaluate identical legal advice when it is attributed either to an AI system or to a human lawyer.<n>We find that attribution to an AI system has no net effect on perceived reasonableness.<n>Providing legal reasoning substantially increases perceived reasonableness regardless of source.
Score: 16.72723020422539
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Abstract: AI systems are increasingly used to provide legal advice, raising questions about whether laypeople accept guidance from algorithms--especially when that advice is legally correct but socially controversial. We report a preregistered survey experiment with 3,348 adults in mainland China examining how people evaluate identical legal advice when it is attributed either to an AI system or to a human lawyer, and when it is accompanied by reasoning or not. Contrary to expectations of algorithm aversion, attribution to an AI system has no net effect on perceived reasonableness. However, mediation analyses reveal opposing psychological pathways underlying this null result. AI-attributed advice is perceived as more objective, which increases perceived reasonableness, but also as less comprehensive and less attentive to special circumstances, which decreases perceived reasonableness. By contrast, providing legal reasoning substantially increases perceived reasonableness regardless of source, largely by enhancing perceptions of objectivity. Qualitative responses corroborate this tension between objectivity and contextual sensitivity in evaluations of legal advice. Together, these findings suggest that public responses to AI legal advisors are shaped not by rigid attitudes toward automation, but by the balancing of competing normative expectations. The results have implications for theories of algorithm aversion and the design of AI recommendation systems in normatively salient domains.

Related papers

Credible, Not Always Correct: How Reddit Users Verify AI-Generated Legal Advice [0.46976113832881716]
Large language models (LLMs) are increasingly used by laypeople to resolve real legal problems.<n>This article presents evidence that the practical force of AI-generated legal advice depends not on its accuracy but on the social production of its credibility.
arXiv Detail & Related papers (2026-08-13T15:33:12Z)
Man and machine: artificial intelligence and judicial decision making [0.0]
We conduct a synthetic review connecting three aspects of AI's role in judicial decision-making.<n>The impact of AI decision-aid tools on pretrial and sentencing decisions is modest or nonexistent.<n>We argue that AI-versus-human comparisons have the potential to yield new insights into both algorithmic tools and human decision-makers.
arXiv Detail & Related papers (2026-03-19T15:38:50Z)
Security, privacy, and agentic AI in a regulatory view: From definitions and distinctions to provisions and reflections [53.468443050795344]
The rapid proliferation of artificial intelligence (AI) technologies has led to a dynamic regulatory landscape.<n>As AI paradigms shift towards greater autonomy, specifically in the form of agentic AI, it becomes increasingly challenging to articulate regulatory stipulations.<n>This paper reviews the evolving European Union (EU) AI regulatory provisions via analyzing 24 relevant documents published between 2024 and 2025.
arXiv Detail & Related papers (2026-03-19T13:50:52Z)
Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics [49.3262123849242]
We introduce LEGIT (LEGal Issue Trees), a novel large-scale (24K instances) expert-level legal reasoning dataset.<n>We convert court judgments into hierarchical trees of opposing parties' arguments and the court's conclusions, which serve as rubrics for evaluating the issue coverage and correctness of the reasoning traces.
arXiv Detail & Related papers (2025-11-30T18:32:43Z)
Argumentation-Based Explainability for Legal AI: Comparative and Regulatory Perspectives [0.9668407688201359]
Artificial Intelligence (AI) systems are increasingly deployed in legal contexts.<n>The so-called black box problem'' undermines legitimacy of automated decision-making.<n>XAI has proposed a variety of methods to enhance transparency.
arXiv Detail & Related papers (2025-10-13T07:19:15Z)
The Silicon Reasonable Person: Can AI Predict How Ordinary People Judge Reasonableness? [0.0]
This Article investigates whether large language models (LLMs) can learn to identify patterns driving human reasonableness judgments.<n>We show that certain models capture not just surface-level responses but potentially their underlying decisional architecture.<n>These findings suggest practical applications: judges could calibrate intuitions against broader patterns, lawmakers could test policy interpretations, and resource-constrained litigants could preview argument reception.
arXiv Detail & Related papers (2025-08-04T06:19:45Z)
AI Debate Aids Assessment of Controversial Claims [73.8907110799657]
We study whether AI debate can guide biased judges toward the truth by having two AI systems debate opposing sides of controversial factuality claims.<n>In Study I, debate consistently improves human judgment accuracy and confidence calibration, outperforming consultancy.<n>In Study II, AI judges with human-like personas achieve even higher accuracy (78.5%) than human judges (70.1%) and default AI judges without personas (69.8%)<n>These findings highlight AI debate as a promising path toward scalable, bias-resilient oversight in contested domains.
arXiv Detail & Related papers (2025-06-02T19:01:53Z)
RLJP: Legal Judgment Prediction via First-Order Logic Rule-enhanced with Large Language Models [58.69183479148083]
Legal Judgment Prediction (LJP) is a pivotal task in legal AI.<n>Existing LJP models integrate judicial precedents and legal knowledge for high performance.<n>But they neglect legal reasoning logic, a critical component of legal judgments requiring rigorous logical analysis.<n>This paper proposes a rule-enhanced legal judgment prediction framework based on first-order logic (FOL) formalism and comparative learning (CL)
arXiv Detail & Related papers (2025-05-27T14:50:21Z)
Must Read: A Systematic Survey of Computational Persuasion [60.83151988635103]
AI-driven persuasion can be leveraged for beneficial applications, but also poses threats through manipulation and unethical influence.<n>Our survey outlines future research directions to enhance the safety, fairness, and effectiveness of AI-powered persuasion.
arXiv Detail & Related papers (2025-05-12T17:26:31Z)
A Critical Examination of the Ethics of AI-Mediated Peer Review [0.0]
Recent advancements in artificial intelligence (AI) systems offer promise and peril for scholarly peer review. Human peer review systems are also fraught with related problems, such as biases, abuses, and a lack of transparency. The legitimacy of AI-driven peer review hinges on the alignment with the scientific ethos.
arXiv Detail & Related papers (2023-09-02T18:14:10Z)
Fairness in Agreement With European Values: An Interdisciplinary Perspective on AI Regulation [61.77881142275982]
This interdisciplinary position paper considers various concerns surrounding fairness and discrimination in AI, and discusses how AI regulations address them. We first look at AI and fairness through the lenses of law, (AI) industry, sociotechnology, and (moral) philosophy, and present various perspectives. We identify and propose the roles AI Regulation should take to make the endeavor of the AI Act a success in terms of AI fairness concerns.
arXiv Detail & Related papers (2022-06-08T12:32:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.