Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions

• URL: http://arxiv.org/abs/2605.24452v1
• Date: Sat, 23 May 2026 08:01:14 GMT
• Title: Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions
• Abstract summary: Legal NLP benchmarks evaluate models on randomly split data, implicitly assuming that legal language is stationary.<n>We test this assumption by fine-tuning four transformer encoders on Ukrainian court decisions from three temporal epochs.<n>Each model is trained on one epoch and evaluated on all three, producing a 3x3 cross-temporal generalization matrix.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Legal NLP benchmarks evaluate models on randomly split data, implicitly assuming that legal language is stationary. We test this assumption by fine-tuning four transformer encoders -- XLM-RoBERTa (base and large) and their legal-domain variants -- on Ukrainian court decisions from three temporal epochs defined by geopolitical disruptions: pre-war (2008-2013), hybrid war (2014-2021), and full-scale invasion (2022-2026). Each model is trained on one epoch and evaluated on all three, producing a 3x3 cross-temporal generalization matrix. Four findings emerge. (1) Forward degradation is severe: models trained on pre-war data lose up to 27.2 percentage points of macro-F1 when applied to full-scale invasion era decisions. (2) The degradation is asymmetric: backward transfer (full-scale to pre-war) is substantially more robust than forward transfer, consistent with the hypothesis that legal language is additive. (3) Legal-domain pretraining (Legal-XLM-R) does not improve absolute performance but reduces forward degradation magnitude and asymmetry. (4) Chronological continual learning eliminates catastrophic forgetting for general XLM-R: pre-war knowledge is fully retained (+1.8 to +6.2 pp) while full-scale performance gains +16.5 to +19.0 pp; reverse-chronological training causes severe forgetting. Cross-jurisdictional pretraining on Swiss Judgment Prediction data improves absolute performance but does not reduce temporal degradation magnitude, confirming that temporal drift is an intrinsic property of legal language evolution. The dataset (428K decisions across three epochs) is publicly available as a LEXTREME contribution.

Related papers

• FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models [50.86041384789834]
  We propose a two-stage PTQ framework for dLLMs that prioritizes the protection of fragile frontier states without requiring end-to-end diffusion rollouts.<n>We show that FAIR-Calib consistently outperforms state-of-the-art baselines on LLaDA and Dream (W4A4), reducing frontier decision flips and suppressing post-commit mismatches across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-06-04T08:00:51Z)
• ESPO: Early-Stopping Proximal Policy Optimization [78.79610718910628]
  ESPO (Early-Stopping Proximal Policy Optimization) detects trajectory failure on-the-fly and terminates rollouts early.<n>On DeepSeek-R1-Distill-Qwen-7B trained for mathematical reasoning, ESPO surpasses PPO on AIME2024 (46.28% vs. 45.25%), AMC2023 (85.83% vs. 82.94%), and MATH-500 (87.42% vs. 85.43%)
  arXiv  Detail & Related papers  (2026-05-28T12:40:22Z)
• Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions [0.0]
  Multi-Legal-Bench is the first cross-jurisdictional legal benchmark that evaluates identical tasks across six countries.<n> benchmark defines five tasks court-type classification, judgment form classification, case-outcome prediction, legal norm extraction, and cause category prediction.<n>We evaluate 7 frontier LLMs under zero-shot and 3-shot prompting via AWS Bedrock, with 4 additional small/medium models (3-12B) for scaling analysis.
  arXiv  Detail & Related papers  (2026-05-28T10:31:37Z)
• UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning [0.0]
  We introduce UA-Legal-Bench, a five-task benchmark for evaluating large language models on Ukrainian legal reasoning.<n>We evaluate 11 LLMs from five families under zero-shot and 3-shot prompting via AWS Bedrock with 158K API calls.<n>Our results reveal sharply task-dependent few-shot effects: few-shot prompting improves judgment form classification by up to +38.6 pp but has mixed effects on outcome prediction.
  arXiv  Detail & Related papers  (2026-05-27T23:12:20Z)
• Temporal Decay of Co-Citation Predictability: A 20-Year Statute Retrieval Benchmark from 396M Ukrainian Court Citations [0.0]
  Co-citation structure is widely assumed to provide stable retrieval signal in legal information systems.<n>We test this assumption longitudinally by constructing UA-StatuteRetrieval, a benchmark that measures co-citation predictability across 20 annual snapshots.<n>We find that Adamic-Adar MRR declines 33% on a fixed set of articles and 47% under a train/test temporal split confirming genuine temporal decay.
  arXiv  Detail & Related papers  (2026-05-17T20:15:37Z)
• Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study [0.0]
  Tokenizer fertility varies 1.6x across foundation models on Ukrainian legal text.<n>Qwen 3 models consume 60% more tokens than Llama-family models on identical input.<n>Few-shot degrades performance by up to 26 percentage points.
  arXiv  Detail & Related papers  (2026-05-14T14:35:05Z)
• Semantic Tube Prediction: Beating LLM Data Efficiency with JEPA [50.494504099850325]
  We introduce the Geodesic Hypothesis, positing that token sequences trace geodesics on a smooth semantic manifold and are therefore locally linear.<n>We show this constraint improves signal-to-noise ratio, and preserves diversity by preventing collisions during trajectory.<n>We demonstrate that geometric priors can surpass brute-force scaling.
  arXiv  Detail & Related papers  (2026-02-26T04:45:07Z)
• Leap+Verify: Regime-Adaptive Speculative Weight Prediction for Accelerating Neural Network Training [0.0]
  We introduce Leap+Verify to accelerate neural network training.<n>It applies speculative execution -- predicting future model weights and validating predictions before acceptance.<n>We evaluate Leap+Verify on GPT-2 124M and Qwen 2.5-1.5B trained on WikiText-103 across five random seeds.
  arXiv  Detail & Related papers  (2026-02-23T08:01:44Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)
• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models [54.08784776767683]
  We present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels.<n>PhysCond-WMA induces semantic, logic, or decision-level distortion while preserving perceptual fidelity.
  arXiv  Detail & Related papers  (2026-02-21T07:22:37Z)
• Relative Scaling Laws for LLMs [91.73497548097775]
  Scaling laws describe how language models improve with additional data, parameters, and compute.<n>We introduce relative scaling laws, which track how performance gaps between test distributions evolve with scale.<n>These results show that although scaling improves overall performance, it is not a universal equalizer.
  arXiv  Detail & Related papers  (2025-10-28T16:55:22Z)
• The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws [51.608402959163925]
  We present the first systematic exploration of optimal sparse pre-training configurations for large language models.<n>We find that initiating pruning at 25% of total training compute and concluding at 75% achieves near-optimal final evaluation loss.<n>We propose a new scaling law that modifies the Chinchilla scaling law to use the average parameter count over pre-training.
  arXiv  Detail & Related papers  (2025-01-21T20:23:22Z)
• Beyond Guilt: Legal Judgment Prediction with Trichotomous Reasoning [12.589047235741194]
  We introduce LJPIV, the first benchmark dataset for Legal Judgment Prediction with Innocent Verdicts.<n>Adhering to the trichotomous dogmatics, we extend three widely-used legal datasets through LLM-based augmentation and manual verification.<n>Our experiments with state-of-the-art legal LLMs and novel strategies that integrate trichotomous reasoning into zero-shot prompting and fine-tuning reveal: (1) current legal LLMs have significant room for improvement, with even the best models achieving an F1 score of less than 0.3 on LJPIV.
  arXiv  Detail & Related papers  (2024-12-19T07:14:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.