Annotated Surrogate Retrieval for Polish Statutory Law

• URL: http://arxiv.org/abs/2608.30929v1
• Date: Mon, 31 Aug 2026 15:05:00 GMT
• Title: Annotated Surrogate Retrieval for Polish Statutory Law
• Abstract summary: We present a family of retrieval methods for Polish statutory law built on document surrogates.<n>ASCR is a surrogate cascade with reranking; ASCR-H fuses a dense list into that cascade; and DTF replaces both language-model stages with three lexical and dense retrievers.<n>We evaluate all three against fourteen lexical, dense, fused and ablated baselines plus four controls, on 300 questions from the 2024 and 2025 Polish bar and legal counsel entrance examinations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a family of retrieval methods for Polish statutory law built on document surrogates: language-model annotations attached to statutory articles at index time. Three designs occupy different points on the cost-quality frontier. ASCR is a surrogate cascade with reranking; ASCR-H fuses a dense list into that cascade; and DTF replaces both language-model stages with three lexical and dense retrievers, weighted reciprocal rank fusion, and a deterministic re-scoring prior, using no model call before generation. We evaluate all three against fourteen lexical, dense, fused and ablated baselines plus four controls, on 300 questions from the 2024 and 2025 Polish bar and legal counsel entrance examinations (264 with their reference article in the corpus), over 82,508 articles from 1,133 acts. On paired McNemar tests, ASCR-H places the reference provision at rank one significantly more often than every other non-oracle configuration except one of its own ablations (eighteen of twenty comparisons significant in its favour at p < 0.005), reaching 72.3% against 61.7% for BM25 and 52.3% for dense retrieval. The advantage is concentrated at the head and does not survive depth: it is significant at cutoffs of one and five, disappears by ten, and by twenty DTF leads on point estimate (86.0% versus 84.5%) at one ninth the latency and less than half the cost. Ablation attributes 27.6 points of rank-one accuracy to the reranking stage alone. We further report that the ranking advantage does not extend to citation accuracy, where DTF matches the oracle ceiling, and three negative results on lemmatisation, pseudo-relevance feedback and query rewriting. Surrogate annotation covers 27.0% of the corpus but every reference provision in the benchmark, an asymmetry we disclose and discuss. Benchmark, per-question outputs and paired significance tests are publicly available.

Related papers

• State of Thought Enables Endogenous Reasoning [48.902117018115256]
  Test-time reasoning has emerged as a major approach to improving the capabilities of Large Language Models (LLMs)<n>We propose State of Thought, a new reasoning paradigm that enables endogenous reasoning in LLMs.<n>We show that SoT consistently improves mean-baseline accuracy while reducing generated tokens by 62.6% and end-to-end latency by 44.6%.
  arXiv  Detail & Related papers  (2026-09-13T05:55:03Z)
• From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop [63.00693129203923]
  The Workshop on Trustworthy Natural Language Processing has grown from 8 proceedings papers to 41 over six editions.<n>We synthesize insights from all 144 proceedings papers, classifying them along six trust dimensions grounded in established frameworks.<n>Analysis from the classification study reveals that truthfulness is the fastest-growing dimension (absent in 2021-2022, comprising 37% of papers by 2025-2026)
  arXiv  Detail & Related papers  (2026-08-11T17:30:16Z)
• Temporal Misgrounding in Legal RAG: A Versioned-Corpus Benchmark for French Tax Law [0.0]
  We identify and quantify temporal misgrounding: the systematic retrieval and citation of the currently in-force version of a legal article.<n>We introduce FiscalQA Pro, pairing a versioned corpus of 32,436 article-versions of the French tax code with an all-model-hard temporal-reasoning track.<n>Our end-to-end retriever over a multi-version index, with no oracle, reaches 98.3% mean strict.
  arXiv  Detail & Related papers  (2026-08-10T10:20:13Z)
• Knowledge Index of Noah's Ark [63.143852586221534]
  We introduce KINA, an 899-item benchmark across 261 fine-grained disciplines.<n>We prove a bonus-on-bar tournament weakly FOSD-dominates flat payment in released-review quality.<n>The top model, Gemini-3.1-Pro-Preview, reaches 53.17%, followed by Claude-Opus-4.6 at 49.92% and GPT-5.4 at 48.55%.
  arXiv  Detail & Related papers  (2026-06-03T17:06:49Z)
• Benchmarking Speech-to-Speech Translation Models [55.00303727199927]
  Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol.<n>We introduce a benchmarking framework integrating 46 metrics across eight dimensions.<n>We deploy it on 1,248 model-language configurations from FLEURS and CVSS.
  arXiv  Detail & Related papers  (2026-06-02T07:01:33Z)
• Step-wise Rubric Rewards for LLM Reasoning [72.17879367869503]
  Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning in large language models.<n>We find 18.2% of steps in correct responses are wrong yet positively rewarded.<n>Across six mathematical reasoning benchmarks, SRaR improves average accuracy over RaR by 3.57 points.
  arXiv  Detail & Related papers  (2026-05-17T07:08:14Z)
• Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study [0.0]
  Tokenizer fertility varies 1.6x across foundation models on Ukrainian legal text.<n>Qwen 3 models consume 60% more tokens than Llama-family models on identical input.<n>Few-shot degrades performance by up to 26 percentage points.
  arXiv  Detail & Related papers  (2026-05-14T14:35:05Z)
• Swiss-Bench SBP-002: A Frontier Model Comparison on Swiss Legal and Regulatory Tasks [0.0]
  No existing benchmark evaluates frontier model performance on applied Swiss regulatory tasks.<n>I introduce Swiss-Bench SBP, a trilingual benchmark of 395 expert-crafted items spanning three Swiss regulatory domains.<n>I evaluate ten frontier models from March 2026 using a structured three-dimension scoring framework.
  arXiv  Detail & Related papers  (2026-03-24T18:41:07Z)
• CORE: Comprehensive Ontological Relation Evaluation for Large Language Models [0.9668495520241466]
  Large Language Models (LLMs) perform well on many reasoning benchmarks, yet existing evaluations rarely assess their ability to distinguish between meaningful semantic relations and genuine unrelatedness.<n>We introduce CORE (Comprehensive Ontological Relation Evaluation), a dataset of 225K multiple-choice questions spanning 74 disciplines.<n>A human baseline from 1,000+ participants achieves 92.6% accuracy (95.1% on unrelated pairs)
  arXiv  Detail & Related papers  (2026-02-06T07:16:33Z)
• An Auditable Pipeline for Fuzzy Full-Text Screening in Systematic Reviews: Integrating Contrastive Semantic Highlighting and LLM Judgment [0.0]
  Full-text screening is the major bottleneck of systematic reviews.<n>We present a scalable, auditable pipeline that reframes inclusion/exclusion as a fuzzy decision problem.
  arXiv  Detail & Related papers  (2025-08-17T17:41:50Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.