SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification

• URL: http://arxiv.org/abs/2608.08786v1
• Date: Sun, 09 Aug 2026 16:05:16 GMT
• Title: SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification
• Abstract summary: Large language models (LLMs) increasingly serve as data-driven reasoners, yet their chains-of-thought (CoT) can be unfaithful even when final answers are correct.<n>We propose textbfSymDiag, a neuro-symbolic framework that textbfreframes reasoning verification as structured failure diagnosis.
• Score: 25.97411184140887
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) increasingly serve as data-driven reasoners, yet their chains-of-thought (CoT) can be unfaithful even when final answers are correct. Most existing ``verification'' signals are not diagnostic: answer matching observes only the outcome, LLM-as-judge provides subjective and non-verifiable critiques, and scalar rewards (e.g., PRMs/RMs) offer little insight into where a multi-step derivation fails.We propose \textbf{SymDiag}, a neuro-symbolic framework that \textbf{reframes reasoning verification as structured failure diagnosis}. SymDiag translates natural-language CoT into symbolic constraints and performs step-level satisfiability/entailment checks to (i) localize failing steps and (ii) produce verifiable diagnostic evidence, including counterexamples, inconsistency witnesses, and missing-premise indicators. A central challenge is that apparent ``logic violations'' can be caused either by genuine reasoning defects or by neural-to-symbolic translation noise. SymDiag therefore incorporates a Self-Auditor that disentangles TranslationError from ReasoningError via dual symbolic encodings consistency checks, enabling robust diagnosis under partial observability. Across diverse mathematical, logical, scientific, and general reasoning benchmarks, SymDiag improves detection of unfaithful reasoning and provides substantially more effective feedback for multi-round reasoning repair than outcome-only verification and LLM-based judging, offering a principled foundation for trustworthy and scalable reasoning diagnosis.

Related papers

• LogSemFuse: Semantic Evidence Fusion for Explainable Log Anomaly Detection [6.950497660953971]
  LogSemFuse is an evidence-guided plug-in framework that enhances existing backbone detectors without replacing their original pipelines.<n>We evaluate LogSemFuse on HDFS, BGL, and Liberty using DeepLog, LogAnomaly, LogBERT, and NeuralLog as backbones.
  arXiv  Detail & Related papers  (2026-07-03T20:50:07Z)
• Uncertainty Reasoning with Large Language Models for Explainable Disease Diagnosis [12.157855328093055]
  Clinical decision-making requires reasoning over incomplete, imprecise, and linguistically expressed patient narratives.<n>We propose a neuro-symbolic reasoning framework that aligns large language models with formal logic.<n>Results show performance comparable to state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2026-05-25T08:18:45Z)
• MedCausalX: Adaptive Causal Reasoning with Self-Reflection for Trustworthy Medical Vision-Language Models [10.466505116993451]
  Existing medical chain-of-thought models lack explicit mechanisms to represent and enforce causal reasoning.<n>We propose MedCausalX, an end-to-end framework explicitly models causal reasoning chains in medical VLMs.<n>We show that MedCausalX consistently outperforms state-of-the-art methods, improving diagnostic consistency by +5.4 points, reducing hallucination by over 10 points, and attaining top spatial grounding IoU.
  arXiv  Detail & Related papers  (2026-03-24T11:28:15Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification [12.60121003165514]
  Vision-language models (VLMs) show promise in drafting radiology reports, yet they frequently suffer from logical inconsistencies.<n>Standard lexical metrics heavily penalize clinical paraphrasing and fail to capture these deductive failures.<n>We introduce a neurosymbolic verification framework that deterministically audits the internal consistency of VLM-generated reports.
  arXiv  Detail & Related papers  (2026-02-27T15:49:59Z)
• Uncovering Hidden Correctness in LLM Causal Reasoning via Symbolic Verification [56.51953062869371]
  DoVerifier is a symbolic verifier that checks whether causal expressions are derivable from a given causal graph using rules from do-calculus and probability theory.<n>Our evaluations on synthetic data and causal QA benchmarks show that DoVerifier more accurately captures semantic correctness of causal reasoning traces.
  arXiv  Detail & Related papers  (2026-01-29T03:22:58Z)
• MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models [49.32415342913976]
  We introduce MM-CoT, a diagnostic benchmark designed to probe the visual grounding and logical coherence of CoT reasoning in multimodal models.<n>We evaluate leading vision-language models on MM-CoT and find that even the most advanced systems struggle, revealing a sharp discrepancy between generative fluency and true reasoning fidelity.
  arXiv  Detail & Related papers  (2025-12-09T04:13:31Z)
• Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models [12.270274049887298]
  Reasoning traces can be redundant or logically inconsistent, making them a new source of hallucination.<n>Existing hallucination detection methods focus primarily on answer-level uncertainty.<n>We propose RACE, a novel framework specifically tailored for hallucination detection in LRMs.
  arXiv  Detail & Related papers  (2025-06-05T09:54:04Z)
• JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models [51.99046112135311]
  We introduce JustLogic, a synthetically generated deductive reasoning benchmark for rigorous evaluation of Large Language Models (LLMs)<n>JustLogic is highly complex, capable of generating a diverse range of linguistic patterns, vocabulary, and argument structures.<n>Our experimental results reveal that (i) state-of-the-art (SOTA) reasoning LLMs perform on par or better than the human average but significantly worse than the human ceiling.
  arXiv  Detail & Related papers  (2025-01-24T15:49:10Z)
• Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs [95.07757789781213]
  Two lines of approaches are adopted for complex reasoning with LLMs.<n>One line of work prompts LLMs with various reasoning structures, while the structural outputs can be naturally regarded as intermediate reasoning steps.<n>The other line of work adopt LLM-free declarative solvers to do the reasoning task, rendering higher reasoning accuracy but lacking interpretability due to the black-box nature of the solvers.<n>We present a simple extension to the latter line of work. Specifically, we showcase that the intermediate search logs generated by Prolog interpreters can be accessed and interpreted into human-readable reasoning.
  arXiv  Detail & Related papers  (2023-11-16T11:26:21Z)
• A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning [73.77088902676306]
  We take a closer look at the self-verification abilities of large language models (LLMs) in the context of logical reasoning. Our main findings suggest that existing LLMs could struggle to identify fallacious reasoning steps accurately and may fall short of guaranteeing the validity of self-verification methods.
  arXiv  Detail & Related papers  (2023-11-14T07:13:10Z)
• Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation [85.12429517510311]
  LogicDiag is a neural-logic semi-supervised learning framework for semantic segmentation. Our key insight is that conflicts within pseudo labels, identified through symbolic knowledge, can serve as strong yet commonly ignored learning signals. We showcase the practical application of LogicDiag in the data-hungry segmentation scenario, where we formalize the structured abstraction of semantic concepts as a set of logic rules.
  arXiv  Detail & Related papers  (2023-08-24T06:50:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.