Which Models Perform Better in Inheritance Reasoning?

• URL: http://arxiv.org/abs/2606.13751v2
• Date: Mon, 15 Jun 2026 08:04:24 GMT
• Title: Which Models Perform Better in Inheritance Reasoning?
• Abstract summary: This paper presents the participation of team PSL in the QIAS 2026 Shared Task on Arabic Islamic inheritance reasoning.<n>The task evaluates the ability of large language models to solve inheritance cases that require legal interpretation, multi-step reasoning, and precise numerical computation.<n>We compare textitcommercial and textitopen-source models under a unified prompting strategy to assess their effectiveness in structured legal reasoning with minimal task-specific adaptation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper presents the participation of team PSL in the QIAS 2026 Shared Task on Arabic Islamic inheritance reasoning. The task evaluates the ability of large language models to solve inheritance cases that require legal interpretation, multi-step reasoning, and precise numerical computation. We compare \textit{commercial} and \textit{open-source} models under a unified prompting strategy to assess their effectiveness in structured legal reasoning with minimal task-specific adaptation. \\ Our results show a clear gap in reliability between the two model families. Commercial models demonstrate stronger performance in identifying eligible heirs, applying exclusion rules, and maintaining consistency across reasoning steps. In contrast, open-source models exhibit greater instability, particularly in cases involving dependent legal decisions and fractional share adjustments. The best performance is achieved by \textit{Gemini 2.5 Flash}, with an MRE of $0.989$.

Related papers

• INSPIRE: An Internalize-Then-Improve Approach for Example-Driven Mathematical Reasoning [61.60016491769943]
  In human mathematics education, example-based reasoning such as constructing counterexamples to test theorem boundaries reflects deep conceptual understanding.<n>We propose INSPIRE, an Internalize-Then-Improve approach combining Reference-Guided Student Internalization (RGSI) with a stage-wise preference training strategy.<n>Experiments across multiple model scales and families demonstrate consistent improvements, even surpassing larger open-source models.
  arXiv  Detail & Related papers  (2026-08-27T03:24:03Z)
• Information-Theoretic Limits of Reliability and Scaling in Language Models [6.658166782006281]
  Large language models (LLMs) are evaluated as though perfect reliability is achievable for any task given sufficient scale.<n>We show this assumption is information-theoretically unjustified.<n>Our work formalizes the resource-complexity tradeoffs that govern model performance across domains.
  arXiv  Detail & Related papers  (2026-05-08T06:06:18Z)
• QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning [1.0152838128195467]
  We present QU-NLP's submission to the QIAS 2026 shared task on Arabic Islamic inheritance reasoning.<n>Our approach employs a multi-stage Quantized Low-Rank Adaptation strategy on Qwen3-4B.<n>Using 4-bit NF4 quantization with rank-128 LoRA, our model achieves 90% MIR-E (Mawarith Inheritance Reasoning Evaluation) score on the test set.
  arXiv  Detail & Related papers  (2026-03-29T09:11:17Z)
• Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models [50.99097734404912]
  We show that RL-friendly models exhibit intra-class compactness and inter-class separation in their probability assignments to correct vs. incorrect responses.<n>Experiments across six mathematical benchmarks show consistent improvements across all model families, with gains up to 5.9 points on AIME24.
  arXiv  Detail & Related papers  (2026-01-11T13:34:44Z)
• Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models [72.4149653187766]
  We propose a Reasoner-Verifier framework named Adrialversa Reasoning RAG (ARR)<n>The Reasoner and Verifier engage in reasoning on retrieved evidence and critiquing each other's logic while being guided by process-aware advantage.<n> Experiments on multiple benchmarks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2026-01-08T06:57:03Z)
• Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluation [0.17592522344393483]
  o3 and Gemini 2.5 achieved accuracies above 90%, whereas ALLaM, Fanar, LLaMA, and Mistral scored below 50%.<n>We conduct a detailed error analysis to identify recurring failure patterns across models.<n>Our findings highlight limitations in handling structured legal reasoning and suggest directions for improving performance in Islamic legal reasoning.
  arXiv  Detail & Related papers  (2025-09-01T03:08:10Z)
• Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability [70.4107059502882]
  Training language models with rationales augmentation has been shown to be beneficial in many existing works.<n>We conduct comprehensive investigations to thoroughly inspect the impact of rationales on model performance.
  arXiv  Detail & Related papers  (2025-05-30T02:39:37Z)
• Reasoning Towards Fairness: Mitigating Bias in Language Models through Reasoning-Guided Fine-Tuning [12.559028963968247]
  We investigate the crucial relationship between a model's reasoning ability and fairness.<n>We find that larger models with stronger reasoning abilities exhibit substantially lower stereotypical bias.<n>We introduce ReGiFT, a novel approach that extracts structured reasoning traces from advanced reasoning models and infuses them into models that lack such capabilities.
  arXiv  Detail & Related papers  (2025-04-08T03:21:51Z)
• Efficient Inference for Large Reasoning Models: A Survey [74.17203483365171]
  Large Reasoning Models (LRMs) significantly improve the reasoning ability of Large Language Models (LLMs) by learning to reason.<n>However, their deliberative reasoning process leads to inefficiencies in token usage, memory consumption, and inference time.<n>This survey provides a review of efficient inference methods designed specifically for LRMs, focusing on mitigating token inefficiency while preserving the reasoning quality.
  arXiv  Detail & Related papers  (2025-03-29T13:27:46Z)
• Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guarantees [3.4289478404209826]
  Large Language Models excel in generative tasks but exhibit inefficiencies in structured text selection.<n>We propose a Learning-to-Defer framework that allocates queries to specialized experts, ensuring high-confidence predictions.
  arXiv  Detail & Related papers  (2024-10-21T08:21:00Z)
• Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement [3.537369004801589]
  We study the classification of legal reasoning according to jurisprudential philosophy. We use a novel dataset of historical United States Supreme Court opinions annotated by a team of domain experts. We find that generative models perform poorly when given instructions equal to the instructions presented to human annotators.
  arXiv  Detail & Related papers  (2023-10-27T19:27:59Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.