Evaluating RAG for French immigration law: a benchmark and baseline study

• URL: http://arxiv.org/abs/2607.24449v1
• Date: Mon, 27 Jul 2026 13:55:35 GMT
• Title: Evaluating RAG for French immigration law: a benchmark and baseline study
• Abstract summary: International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks.<n>We present a publicly available benchmark and first comparative evaluation for this domain, covering permit-type recommendation, required-document retrieval, and legal citation coverage.
• Score: 0.3058685580689604
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: International recruitment in France requires navigating a layered legal framework absent from existing legal AI benchmarks. We present a publicly available benchmark and first comparative evaluation for this domain, covering permit-type recommendation, required-document retrieval, and legal citation coverage. Comparing a parametric LLM baseline against dense retrieval augmentation at two model scales (Qwen3.5-9B and -27B) on 52 annotated synthetic profiles, we find that retrieval improves administrative guidance at both scales, most notably permit-type accuracy. Our results confirm that retrieval grounding is important for more reliable administrative guidance in this domain, and motivate further investigation of hybrid retrieval strategies.

Related papers

• Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering [54.97384993676565]
  We formalize Regulatory Compliance QA with RegOps-Bench, a novel benchmark featuring an Operational Knowledge Graph derived from complex national R&D regulations.<n>RefWalk traverses cross-document citations, fuses multi-view candidates via max-based aggregation, and enforces per-rule attribution to explicitly map claims to sources.
  arXiv  Detail & Related papers  (2026-05-28T10:38:38Z)
• JUÁ -- A Benchmark for Information Retrieval in Brazilian Legal Text Collections [3.6360100966162787]
  Legal information retrieval in Portuguese remains difficult to evaluate systematically because available datasets differ widely in document type, query style, and relevance definition.<n>We present JU, a public benchmark for Brazilian legal retrieval designed to support more reproducible and comparable evaluation across heterogeneous legal collections.
  arXiv  Detail & Related papers  (2026-04-07T17:10:54Z)
• Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documents [4.735657207313987]
  Retrieval-Augmented Generation (RAG) has emerged as a promising technology for legal document consultation.<n>Existing benchmarks lack specialized support for joint retriever-generator evaluation.<n>LegRAG outperforms existing state-of-the-art methods by 1.3% to 5.6% across key evaluation metrics.
  arXiv  Detail & Related papers  (2026-03-12T10:27:40Z)
• LawThinker: A Deep Research Legal Agent in Dynamic Environments [51.782293183431676]
  LawThinker is an autonomous legal research agent.<n>It enforces verification as an atomic operation after every knowledge exploration step.<n>LawThinker achieves a 24% improvement over direct reasoning.
  arXiv  Detail & Related papers  (2026-02-12T15:19:11Z)
• LegalOne: A Family of Foundation Models for Reliable Legal Reasoning [54.57434222018289]
  We present LegalOne, a family of foundational models specifically tailored for the Chinese legal domain.<n>LegalOne is developed through a comprehensive three-phase pipeline designed to master legal reasoning.<n>We publicly release the LegalOne weights and the LegalKit evaluation framework to advance the field of Legal AI.
  arXiv  Detail & Related papers  (2026-01-31T10:18:32Z)
• ClaimGen-CN: A Large-scale Chinese Dataset for Legal Claim Generation [56.79698529022327]
  Legal claims refer to the plaintiff's demands in a case and are essential to guiding judicial reasoning and case resolution.<n>This paper explores the problem of legal claim generation based on the given case's facts.<n>We construct ClaimGen-CN, the first dataset for Chinese legal claim generation task.
  arXiv  Detail & Related papers  (2025-08-24T07:19:25Z)
• Augmented Question-guided Retrieval (AQgR) of Indian Case Law with LLM, RAG, and Structured Summaries [0.0]
  This paper proposes the use of Large Language Models (LLMs) to facilitate the retrieval of relevant cases.<n>Our approach combines Retrieval Augmented Generation (RAG) with structured summaries optimized for Indian case law.<n>The system generates targeted legal questions based on factual scenarios to identify relevant case law more effectively.
  arXiv  Detail & Related papers  (2025-07-23T05:24:44Z)
• Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Use [44.99833362998488]
  This paper presents a domain-specific implementation of Retrieval-Augmented Generation tailored to the Fair Use Doctrine in U.S. copyright law.<n>Motivated by the increasing prevalence of DMCA takedowns and the lack of accessible legal support for content creators, we propose a structured approach that combines semantic search with legal knowledge graphs and court citation networks to improve retrieval quality and reasoning reliability.
  arXiv  Detail & Related papers  (2025-05-04T15:53:49Z)
• Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study [9.30538764385435]
  Large Language Models (LLMs) have demonstrated strong potential across legal tasks, yet the problem of legal citation prediction remains under-explored.<n>We introduce the AusLaw Citation Benchmark, a real-world dataset comprising 55k Australian legal instances and 18,677 unique citations.<n>We then conduct a systematic benchmarking across a range of solutions.<n>Results show that neither general nor law-specific LLMs suffice as stand-alone solutions, with performance near zero.
  arXiv  Detail & Related papers  (2024-12-09T07:46:14Z)
• JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking [81.88787401178378]
  We introduce JudgeRank, a novel agentic reranker that emulates human cognitive processes when assessing document relevance. We evaluate JudgeRank on the reasoning-intensive BRIGHT benchmark, demonstrating substantial performance improvements over first-stage retrieval methods. In addition, JudgeRank performs on par with fine-tuned state-of-the-art rerankers on the popular BEIR benchmark, validating its zero-shot generalization capability.
  arXiv  Detail & Related papers  (2024-10-31T18:43:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.