LawLuo: A Chinese Law Firm Co-run by LLM Agents

• URL: http://arxiv.org/abs/2407.16252v2
• Date: Sun, 4 Aug 2024 15:27:28 GMT
• Title: LawLuo: A Chinese Law Firm Co-run by LLM Agents
• Authors: Jingyun Sun, Chengxiao Dai, Zhongze Luo, Yangbo Chang, Yang Li,
• Abstract summary: Large Language Models (LLMs) deliver legal consultation services to users without a legal background. Existing Chinese legal LLMs limit interaction to a single model-user dialogue. We propose a novel legal dialogue framework that leverages the collaborative capabilities of multiple LLM agents, termed LawLuo.
• Score: 1.9857357818932064
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) demonstrate substantial potential in delivering legal consultation services to users without a legal background, attributed to their superior text comprehension and generation capabilities. Nonetheless, existing Chinese legal LLMs limit interaction to a single model-user dialogue, unlike the collaborative consultations typical of law firms, where multiple staff members contribute to a single consultation. This limitation prevents an authentic consultation experience. Additionally, extant Chinese legal LLMs suffer from critical limitations: (1) insufficient control over the quality of instruction fine-tuning data; (2) increased model hallucination resulting from users' ambiguous queries; and (3) a reduction in the model's ability to follow instructions over multiple dialogue turns. In response to these challenges, we propose a novel legal dialogue framework that leverages the collaborative capabilities of multiple LLM agents, termed LawLuo. This framework encompasses four agents: a receptionist, a lawyer, a secretary, and a boss, each responsible for different functionalities, collaboratively providing a comprehensive legal consultation to users. Additionally, we constructed two high-quality legal dialogue datasets, KINLED and MURLED, and fine-tuned ChatGLM-3-6b using these datasets. We propose a legal query clarification algorithm called ToLC. Experimental results demonstrate that LawLuo outperforms baseline LLMs, including GPT-4, across three dimensions: lawyer-like language style, the usefulness of legal advice, and the accuracy of legal knowledge. Our code and datasets are available at https://github.com/NEFUJing/LawLuo.

Related papers

• LegalAgentBench: Evaluating LLM Agents in Legal Domain [53.70993264644004]
  LegalAgentBench is a benchmark specifically designed to evaluate LLM Agents in the Chinese legal domain. LegalAgentBench includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge.
  arXiv  Detail & Related papers  (2024-12-23T04:02:46Z)
• AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents [26.268925743738855]
  We present a simulation system called AgentCourt that simulates the entire courtroom process. Our core goal is to enable lawyer agents to learn how to argue a case, as well as improving their overall legal skills. Experiments show that after two lawyer-agents have engaged in a thousand adversarial legal cases in AgentCourt, the evolved lawyer agents exhibit consistent improvement in their ability to handle legal tasks.
  arXiv  Detail & Related papers  (2024-08-15T11:33:20Z)
• DeliLaw: A Chinese Legal Counselling System Based on a Large Language Model [16.63238943983347]
  DeliLaw is a Chinese legal counselling system based on a large language model. Users can consult professional legal questions, search for legal articles and relevant judgement cases, etc. on the DeliLaw system in a dialogue mode.
  arXiv  Detail & Related papers  (2024-08-01T07:54:52Z)
• InternLM-Law: An Open Source Chinese Legal Large Language Model [72.2589401309848]
  InternLM-Law is a specialized LLM tailored for addressing diverse legal queries related to Chinese laws. We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries. InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks.
  arXiv  Detail & Related papers  (2024-06-21T06:19:03Z)
• (A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice [8.48013392781081]
  Large language models (LLMs) are increasingly capable of providing users with advice in a wide range of professional domains, including legal advice. We conducted workshops with 20 legal experts using methods inspired by case-based reasoning. Our findings reveal novel legal considerations, such as unauthorized practice of law, confidentiality, and liability for inaccurate advice.
  arXiv  Detail & Related papers  (2024-02-02T19:35:34Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [15.98468948605927]
  LegalBench is a benchmark consisting of 162 tasks covering six different types of legal reasoning. This paper describes LegalBench, presents an empirical evaluation of 20 open-source and commercial LLMs, and illustrates the types of research explorations LegalBench enables.
  arXiv  Detail & Related papers  (2023-08-20T22:08:03Z)
• Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language Model [30.30848216845138]
  Chatlaw is an innovative legal assistant utilizing a Mixture-of-Experts (MoE) model and a multi-agent system. By integrating knowledge graphs with artificial screening, we construct a high-quality legal dataset to train the MoE model. Our MoE model outperforms GPT-4 in the Lawbench and Unified Exam Qualification for Legal Professionals by 7.73% in accuracy and 11 points, respectively.
  arXiv  Detail & Related papers  (2023-06-28T10:48:34Z)
• SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval [75.05173891207214]
  Legal case retrieval plays a core role in the intelligent legal system. Most existing language models have difficulty understanding the long-distance dependencies between different structures. We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
  arXiv  Detail & Related papers  (2023-04-22T10:47:01Z)
• Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents [56.40163943394202]
  We release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding. We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering.
  arXiv  Detail & Related papers  (2021-05-09T09:39:25Z)
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence [81.04070052740596]
  Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain. This paper introduces the history, the current state, and the future directions of research in LegalAI.
  arXiv  Detail & Related papers  (2020-04-25T14:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.