NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO

• URL: http://arxiv.org/abs/2607.03957v1
• Date: Sat, 04 Jul 2026 17:11:21 GMT
• Title: NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO
• Abstract summary: NormWorlds-CF is a solver-verified environment for counterfactual normative reasoning in rule worlds.<n>Its deterministic solver produces final answers, proof and falsification certificates, argument statuses, support sets, and paired-world change labels.
• Score: 1.5483078145498086
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language models can reach the right normative verdict for the wrong reason. We introduce NormWorlds-CF, a solver-verified environment for counterfactual normative reasoning in executable rule worlds. Its deterministic solver produces final answers, proof and falsification certificates, argument statuses, support sets, and paired-world change labels, enabling supervision and evaluation without LLM judges. The benchmark contains staged SFT diagnostics and a compact paired-world task with 270 root families and 1080 canonical-to-variant pairs. The SFT diagnostics show that final-answer supervision is an unsafe proxy: answer-only SFT reaches perfect accuracy on answer tasks but scores zero on falsification, while proof-plus-falsification training with targeted replay reaches strong all-task accuracy. For the structured-change task, we introduce metamorphic-relation GRPO (MR-GRPO), a class-conditioned reward for GRPO that gives partial credit for relation families and solver-visible change fields. In matched 1.7B continuation experiments, MR-GRPO improves held-out relation accuracy and relation-family correctness, and reduces wrong-family error, compared to sparse and answer-only GRPO. In Qwen3-4B three-seed validation, answer-only reward improves answer-change fields but weakens relation-family structure, sparse reward preserves coarse relation labels best, and MR-GRPO delivers the strongest balanced performance across answer-change, support-change, status-change, and soft root-level metamorphic-relation metrics. These results show that verified counterfactual structure can shape post-training beyond final answers, while exact full change-record generation, invariant subtype recognition, and out-of-distribution (OOD) transfer remain open problems.

Related papers

• DCGC: Draft-Conditioned Global Correction for Complex Reasoning with Masked Diffusion Models [5.2113270099125915]
  Masked Diffusion Model (MDM) framework for global correction uses an imperfect solution draft from an upstream solver as auxiliary context.<n>DCGC combines task-specificSupervised Fine-Tuning (SFT) with a novel inference-time mechanism called Dynamic Dual-CFG.
  arXiv  Detail & Related papers  (2026-08-26T06:35:26Z)
• INFUSER: Influence-Guided Self-Evolution Improves Reasoning [54.101135873140066]
  We introduce an iterative co-training framework with two co-evolving roles.<n>A solver is trained with standard correctness rewards against the generator-provided answers.<n>An 8B INF co-evolving generator outperforms a frozen 32B thinking generator on math and coding.
  arXiv  Detail & Related papers  (2026-06-08T05:40:36Z)
• Grounded Decoding: Retrieval-Anchored Probability Fusion for Faithful RAG [6.324418485299426]
  Large language models may still prioritize parametric knowledge over retrieved information when conflicts arise.<n>We propose a novel training-free decoding framework, emphGrounded Decoding, to improve factual consistency in RAG without modifying model parameters.<n>Our results indicate that probability-level fusion provides a strong and efficient alternative to logit-level intervention methods for faithful RAG decoding.
  arXiv  Detail & Related papers  (2026-05-29T23:47:39Z)
• LC-ERD: Mining Latent Logic for Self-Evolving Reasoning via Consistency-Regulated Reward Decomposition [55.572260012037084]
  We introduce LC-ERD (Logic-Consistent Endogenous Reward Decomposition), a framework framing self-alignment as latent structure mining.<n>We derive a Variational Logic Potential by aggregating consensus from the model's Latent Logic Expertise.<n>Experiments show LC-ERD delivers a robust self-evolution path, uncovering trade-offs between logic consistency and accuracy.
  arXiv  Detail & Related papers  (2026-05-19T07:27:50Z)
• Interactive Critique-Revision Training for Reliable Structured LLM Generation [18.00222080273147]
  We propose DPA-GRPO, a paired-action training method for a two-player generator--verifier game with structured verifier interventions.<n>We analyze the unregularized game and show that positive probability on strictly lower-reward intervention or revision actions creates a profitable unilateral deviation.<n>Experiments on TaxCalc TY24 show that DPA-GRPO improves structured decision accuracy over zero-shot generation and generator-only RL baselines.
  arXiv  Detail & Related papers  (2026-05-08T17:00:38Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• VERGE: Formal Refinement and Guidance Engine for Verifiable LLM Reasoning [4.3414302048068745]
  We present a neurosymbolic framework that combines Large Language Models with SMT solvers to produce verification-guided answers.<n>We introduce three key innovations: (1) multi-model consensus via formal semantic equivalence checking, (2) semantic routing that directs different claim types to appropriate verification strategies, and (3) precise logical error localization via Minimal Correction Subsets.<n>With the GPT-OSS-120B model, VERGE demonstrates an average performance uplift of 18.7% at convergence across a set of reasoning benchmarks compared to single-pass approaches.
  arXiv  Detail & Related papers  (2026-01-27T20:59:11Z)
• Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense [36.71358559780692]
  HERO is a reinforcement learning framework that integrates verifier signals with reward-model scores in a structured way.<n> HERO consistently outperforms RM-only and verifier-only baselines, with strong gains on both verifiable and hard-to-verify tasks.
  arXiv  Detail & Related papers  (2025-10-08T17:09:41Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward [50.97588334916863]
  We develop CompassVerifier, an accurate and robust lightweight verifier model for evaluation and outcome reward.<n>It demonstrates multi-domain competency spanning math, knowledge, and diverse reasoning tasks, with the capability to process various answer types.<n>We introduce VerifierBench benchmark comprising model outputs collected from multiple data sources, augmented through manual analysis of metaerror patterns to enhance CompassVerifier.
  arXiv  Detail & Related papers  (2025-08-05T17:55:24Z)
• Latent Veracity Inference for Identifying Errors in Stepwise Reasoning [78.29317733206643]
  We introduce Veracity Search (VS), a discrete search algorithm over veracity assignments.<n>It performs otherwise intractable inference in the posterior distribution over latent veracity values.<n>It generalizes VS, enabling accurate zero-shot veracity inference in novel contexts.
  arXiv  Detail & Related papers  (2025-05-17T04:16:36Z)
• Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach [51.012396632595554]
  Invariant representation learning (IRL) encourages the prediction from invariant causal features to labels de-confounded from the environments. Recent theoretical results verified that some causal features recovered by IRLs merely pretend domain-invariantly in the training environments but fail in unseen domains. We develop an approach based on conditional mutual information with respect to RS-SCM, then rigorously rectify the spurious and fake invariant effects.
  arXiv  Detail & Related papers  (2023-12-15T12:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.