Retrieval-Based Multi-Label Legal Annotation: Extensible, Data-Efficient and Hallucination-Free

• URL: http://arxiv.org/abs/2605.16767v1
• Date: Sat, 16 May 2026 02:40:01 GMT
• Title: Retrieval-Based Multi-Label Legal Annotation: Extensible, Data-Efficient and Hallucination-Free
• Abstract summary: Parametric encoders typically require task-specific training and retraining when the label set changes.<n>generative large language models becomes costly and degrades as the label space grows.<n>We cast legal annotation as retrieval: we embed documents and label descriptions with a frozen retrieval model and predict labels via k-nearest neighbors in the embedding space.
• Score: 3.5784933879188796
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multi-label legal annotation requires assigning multiple labels from large, evolving taxonomies to long, fact-intensive documents, often under limited supervision. Parametric encoders typically require task-specific training and retraining when the label set changes, while prompting generative large language models becomes costly and degrades as the label space grows. We cast legal annotation as retrieval: we embed documents and label descriptions with a frozen retrieval model and predict labels via k-nearest neighbors in the embedding space, enabling updates by re-embedding and re-indexing rather than gradient-based backpropagation. Across three legal datasets (ECtHR-A, ECtHR-B, and Eurlex with 100 labels), retrieval achieves competitive accuracy and strong data efficiency; on Eurlex, Qwen-8B retrieval improves Macro-F1 from 40.41 (GPT-5.2, zero-shot) to 49.12 while reducing estimated compute by 20-30 times compared to fine-tuning. With only (N=100) training samples, retrieval nearly doubles Micro-F1 over hierarchical Legal-BERT on ECtHR-A (48.29 vs. 27.87). We also quantify a reliability failure mode of generative inference: GPT-5.2 hallucinates labels outside the provided taxonomy in 0.12-0.9% of test samples under deterministic decoding. In contrast, retrieval strictly respects defined label sets, eliminating hallucination by design. These results suggest retrieval-model-based annotators are a practical, deployable alternative for high-cardinality and rapidly changing legal label spaces.

Related papers

• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling [5.963398606353896]
  Rhetorical Role Labeling (RRL) assigns a functional role to each sentence in a document and is widely used in legal, medical, and scientific domains.<n>We introduce RISE, an inference-time semantic reranking framework that leverages label semantics to refine predictions on hard instances.<n>Experiments on eight domain-specific RRL datasets with seven LMs, including encoder-based and causal architectures, show an average gain of +9.15 macro-F1 points on hard examples.
  arXiv  Detail & Related papers  (2026-05-18T08:03:02Z)
• RAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification [1.2599533416395765]
  We present RAPT, a deployment-oriented retrieval-augmented score thresholding wrapper.<n>RAPT is a model-agnostic wrapper for document representations for similarity search and per label confidence scores.<n>In industrial setting, RAPT achieved its best predictive performance with metric learners, reaching 0.87 Macro-F1.
  arXiv  Detail & Related papers  (2026-05-15T18:31:04Z)
• From Lazy to Prolific: Tackling Missing Labels in Open Vocabulary Extreme Classification by Positive-Unlabeled Sequence Learning [5.979699034554517]
  Open-vocabulary Extreme Multi-label Classification (OXMC) extends traditional XMC by allowing prediction beyond a $103$ to $1012$ label set.<n>Self-selection bias in data annotation leads to significant missing labels in both training and test data.<n>We introduce Positive-Unlabeled Sequence Learning (PUSL), which reframes OXMC as an infinite keyphrase generation task.
  arXiv  Detail & Related papers  (2024-08-16T19:10:48Z)
• Retraining with Predicted Hard Labels Provably Increases Model Accuracy [77.71162068832108]
  Retraining can improve the population accuracy obtained by initially training with the given (noisy) labels.<n>We empirically show that retraining selectively on the samples for which the predicted label matches the given label significantly improves label DP training at no extra privacy cost.
  arXiv  Detail & Related papers  (2024-06-17T04:53:47Z)
• Enhancing chest X-ray datasets with privacy-preserving large language models and multi-type annotations: a data-driven approach for improved classification [0.6144680854063935]
  In chest X-ray (CXR) image analysis, rule-based systems are usually employed to extract labels from reports for dataset releases. We present MAPLEZ, a novel approach leveraging a locally executable Large Language Model (LLM) to extract and enhance findings labels.
  arXiv  Detail & Related papers  (2024-03-06T20:10:41Z)
• Substituting Data Annotation with Balanced Updates and Collective Loss in Multi-label Text Classification [19.592985329023733]
  Multi-label text classification (MLTC) is the task of assigning multiple labels to a given text. We study the MLTC problem in annotation-free and scarce-annotation settings in which the magnitude of available supervision signals is linear to the number of labels. Our method follows three steps, (1) mapping input text into a set of preliminary label likelihoods by natural language inference using a pre-trained language model, (2) calculating a signed label dependency graph by label descriptions, and (3) updating the preliminary label likelihoods with message passing along the label dependency graph.
  arXiv  Detail & Related papers  (2023-09-24T04:12:52Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels [60.675714333081466]
  Multi-label recognition (MLR) with incomplete labels is very challenging. Recent works strive to explore the image-to-label correspondence in the vision-language model, ie, CLIP, to compensate for insufficient annotations. We advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior.
  arXiv  Detail & Related papers  (2023-03-23T12:39:20Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Interaction Matching for Long-Tail Multi-Label Classification [57.262792333593644]
  We present an elegant and effective approach for addressing limitations in existing multi-label classification models. By performing soft n-gram interaction matching, we match labels with natural language descriptions.
  arXiv  Detail & Related papers  (2020-05-18T15:27:55Z)
• Unsupervised Person Re-identification via Multi-label Classification [55.65870468861157]
  This paper formulates unsupervised person ReID as a multi-label classification task to progressively seek true labels. Our method starts by assigning each person image with a single-class label, then evolves to multi-label classification by leveraging the updated ReID model for label prediction. To boost the ReID model training efficiency in multi-label classification, we propose the memory-based multi-label classification loss (MMCL)
  arXiv  Detail & Related papers  (2020-04-20T12:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.