Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models

• URL: http://arxiv.org/abs/2401.01301v2
• Date: Fri, 21 Jun 2024 15:32:27 GMT
• Title: Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
• Abstract summary: Large language models (LLMs) are increasingly being used to augment legal practice, education, and research. We present the first systematic evidence of hallucinations, documenting LLMs' varying performance across jurisdictions, courts, time periods, and cases. We find that legal hallucinations are alarmingly prevalent, occurring between 58% of the time with ChatGPT 4 and 88% with Llama 2.
• Score: 8.04008608493335
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Do large language models (LLMs) know the law? These models are increasingly being used to augment legal practice, education, and research, yet their revolutionary potential is threatened by the presence of hallucinations -- textual output that is not consistent with legal facts. We present the first systematic evidence of these hallucinations, documenting LLMs' varying performance across jurisdictions, courts, time periods, and cases. Our work makes four key contributions. First, we develop a typology of legal hallucinations, providing a conceptual framework for future research in this area. Second, we find that legal hallucinations are alarmingly prevalent, occurring between 58% of the time with ChatGPT 4 and 88% with Llama 2, when these models are asked specific, verifiable questions about random federal court cases. Third, we illustrate that LLMs often fail to correct a user's incorrect legal assumptions in a contra-factual question setup. Fourth, we provide evidence that LLMs cannot always predict, or do not always know, when they are producing legal hallucinations. Taken together, our findings caution against the rapid and unsupervised integration of popular LLMs into legal tasks. Even experienced lawyers must remain wary of legal hallucinations, and the risks are highest for those who stand to benefit from LLMs the most -- pro se litigants or those without access to traditional legal resources.

Related papers

• LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context [0.05999777817331315]
  We find that authority-style prefixes increase refusal rates by 2--20x over the no-play baseline.<n>A known role-play prefix shows mixed effects, sharply increasing refusals in some models and barely shifting them in others.<n>The finding suggests that small on-prem deployable LLMs are unstable under contextual framings that a real institutional user might naturally introduce.
  arXiv  Detail & Related papers  (2026-06-23T13:47:07Z)
• Place Matters: Comparing LLM Hallucination Rates for Place-Based Legal Queries [8.865671688076574]
  We show that the rate of hallucination of legal information by leading closed-source LLMs is significantly associated with place.<n>This suggests that the quality of legal solutions provided by these models is not evenly distributed across geography.
  arXiv  Detail & Related papers  (2025-11-10T04:42:00Z)
• Trust Me, I'm Wrong: High-Certainty Hallucinations in LLMs [45.13670875211498]
  Large Language Models (LLMs) often generate outputs that lack grounding in real-world facts, a phenomenon known as hallucinations. We show that models can hallucinate with high certainty even when they have the correct knowledge.
  arXiv  Detail & Related papers  (2025-02-18T15:46:31Z)
• FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs [2.871226288151562]
  This paper introduces FaithBench, a summarization hallucination benchmark comprising challenging hallucinations made by 10 modern LLMs. Our results show GPT-4o and GPT-3.5-Turbo produce the least hallucinations. Even the best hallucination detection models have near 50% accuracies on FaithBench, indicating lots of room for future improvement.
  arXiv  Detail & Related papers  (2024-10-17T04:30:46Z)
• MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Models [26.464489158584463]
  We conduct a pioneering study of hallucinations in LLM-generated responses to real-world healthcare queries from patients. We propose MedHalu, a carefully crafted first-of-its-kind medical hallucination dataset with a diverse range of health-related topics. We also introduce MedHaluDetect framework to evaluate capabilities of various LLMs in detecting hallucinations.
  arXiv  Detail & Related papers  (2024-09-29T00:09:01Z)
• WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries [64.239202960816]
  We introduce WildHallucinations, a benchmark that evaluates factuality. It does so by prompting large language models to generate information about entities mined from user-chatbot conversations in the wild. We evaluate 118,785 generations from 15 LLMs on 7,919 entities.
  arXiv  Detail & Related papers  (2024-07-24T17:59:05Z)
• On Large Language Models' Hallucination with Regard to Known Facts [74.96789694959894]
  Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.
  arXiv  Detail & Related papers  (2024-03-29T06:48:30Z)
• Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States [19.343629282494774]
  Large Language Models (LLMs) can make up answers that are not real, and this is known as hallucination. This research aims to see if, how, and to what extent LLMs are aware of hallucination.
  arXiv  Detail & Related papers  (2024-02-15T06:14:55Z)
• The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models [134.6697160940223]
  hallucination poses great challenge to trustworthy and reliable deployment of large language models. Three key questions should be well studied: how to detect hallucinations (detection), why do LLMs hallucinate (source), and what can be done to mitigate them. This work presents a systematic empirical study on LLM hallucination, focused on the the three aspects of hallucination detection, source and mitigation.
  arXiv  Detail & Related papers  (2024-01-06T12:40:45Z)
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations [67.35512483340837]
  Large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information. We propose a simple textitInduce-then-Contrast Decoding (ICD) strategy to alleviate hallucinations.
  arXiv  Detail & Related papers  (2023-12-25T12:32:49Z)
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions [40.79317187623401]
  The emergence of large language models (LLMs) has marked a significant breakthrough in natural language processing (NLP) LLMs are prone to hallucination, generating plausible yet nonfactual content. This phenomenon raises significant concerns over the reliability of LLMs in real-world information retrieval systems.
  arXiv  Detail & Related papers  (2023-11-09T09:25:37Z)
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models [116.01843550398183]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks. LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge.
  arXiv  Detail & Related papers  (2023-09-03T16:56:48Z)
• HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models [146.87696738011712]
  Large language models (LLMs) are prone to generate hallucinations, i.e., content that conflicts with the source or cannot be verified by the factual knowledge. To understand what types of content and to which extent LLMs are apt to hallucinate, we introduce the Hallucination Evaluation benchmark for Large Language Models (HaluEval)
  arXiv  Detail & Related papers  (2023-05-19T15:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.