HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning

• URL: http://arxiv.org/abs/2608.02026v1
• Date: Mon, 03 Aug 2026 10:23:48 GMT
• Title: HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning
• Abstract summary: We propose a hypergraph-based paired failure attribution framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path.<n>By reducing the search space, our method efficiently localizes root causes and enables scalable synthesis of attribution data.<n>Experiments on mathematical reasoning and agentic coding tasks demonstrate that HPFA can dramatically increase attribution accuracy and efficiency.
• Score: 15.068861766259412
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reflection is a powerful mechanism for LLM reasoning, yet its effectiveness hinges on accurately attributing failures to specific reasoning steps, a capability that current models notably lack. Existing failure attribution methods either require expensive step-by-step counterfactual testing that scales poorly with trajectory length, or treat reasoning traces as flat sequences that ignore the inherent non-linear logical dependencies. We propose a hypergraph-based paired failure attribution (HPFA) framework that attributes the failure root cause by comparing the hyperedges of the targeted failure reasoning path against a reference successful path. By reducing the search space, our method efficiently localizes root causes and enables scalable synthesis of attribution data for training a lightweight attributor model via supervised fine-tuning and reinforcement learning. Experiments on mathematical reasoning and agentic coding tasks demonstrate that HPFA can dramatically increase attribution accuracy and efficiency, and the trained attributor consistently improves reasoning accuracy at test time, outperforming baselines that lack graph structure or paired analysis.

Related papers

• Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution [47.70578021328887]
  Large language model (LLM)-based multi-agent systems often exhibit complex failure modes.<n>We introduce AFANet, a lightweight graph-based framework that models interaction trajectories through step-level semantic signals and agent-level relationships.<n>Our results suggest that effective agent failure attribution does not require heavy reasoning and a lightweight, structured approach can achieve strong performance.
  arXiv  Detail & Related papers  (2026-08-19T06:16:09Z)
• Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence [70.8951332710039]
  Multimodal large language models (MLLMs) have demonstrated significant potential in complex spatial scene understanding and reasoning tasks.<n>However, their open-ended reasoning process is prone to decision errors and error accumulation, leading to instability in answer quality.<n>We propose an advantage-guided gating framework that dynamically intervenes in and corrects deviations during the reasoning process.
  arXiv  Detail & Related papers  (2026-08-08T07:46:04Z)
• Unified Data Selection for LLM Reasoning [66.0118347918729]
  Training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data.<n>We propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5%) highest-entropy tokens in each reasoning sample.
  arXiv  Detail & Related papers  (2026-05-21T12:21:41Z)
• Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks [42.92210265283373]
  Interpretability tools are increasingly used to analyze failures of Large Language Models (LLMs)<n>We study contrastive, LRP-based attribution as a practical tool for analyzing LLM failures in realistic settings.<n>Our results show that this token-level contrastive attribution can yield informative signals in some failure cases, but is not universally applicable.
  arXiv  Detail & Related papers  (2026-04-20T03:24:11Z)
• On Multi-Step Theorem Prediction via Non-Parametric Structural Priors [50.16583672681106]
  In this work, we explore training-free theorem prediction through the lens of in-context learning (ICL)<n>We propose Theorem Precedence Graphs, which encode temporal dependencies from historical solution traces as directed graphs, and impose explicit topological constraints that effectively prune the search space during inference.<n>Experiments on the FormalGeo7k benchmark show that our method achieves 89.29% accuracy, substantially outperforming ICL baselines and matching state-of-the-art supervised models.
  arXiv  Detail & Related papers  (2026-03-05T06:08:50Z)
• From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems [13.588125015284257]
  We propose CHIEF, a novel framework that transforms chaotic trajectories into a structured hierarchical causal graph.<n> Experiments on Who&When benchmark show that CHIEF outperforms eight strong and state-of-the-art baselines on both agent- and step-level accuracy.
  arXiv  Detail & Related papers  (2026-02-27T06:08:42Z)
• How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns [51.02752099869218]
  Large Language Models (LLMs) display strikingly different generalization behaviors.<n>We introduce a novel benchmark that decomposes reasoning into atomic core skills.<n>We show that RL-tuned models maintain more stable behavioral profiles and resist collapse in reasoning skills, whereas SFT models exhibit sharper drift and overfit to surface patterns.
  arXiv  Detail & Related papers  (2025-12-30T08:16:20Z)
• Verifying Large Language Models' Reasoning Paths via Correlation Matrix Rank [71.09032766271493]
  Large language models (LLMs) are prone to errors and hallucinations.<n>How to check their outputs effectively and efficiently has become a critical problem in their applications.
  arXiv  Detail & Related papers  (2025-10-28T11:01:10Z)
• RationAnomaly: Log Anomaly Detection with Rationality via Chain-of-Thought and Reinforcement Learning [27.235259453535537]
  RationAnomaly is a novel framework that enhances log anomaly detection by synergizing Chain-of-Thought fine-tuning with reinforcement learning.<n>We have released the corresponding resources, including code and datasets.
  arXiv  Detail & Related papers  (2025-09-18T07:35:58Z)
• Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision [43.05159920832912]
  Large language models (LLMs) have shown strong performance in many reasoning benchmarks.<n>LLMs are susceptible to content variations, demonstrating a lack of robust planning or symbolic abstractions.<n>We propose to overcome such limitations by synthesizing high-quality symbolic reasoning trajectories with stepwise pseudo-labels.
  arXiv  Detail & Related papers  (2025-05-26T18:06:39Z)
• Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning [53.25336975467293]
  We present the first theoretical error decomposition analysis of methods such as perplexity and self-consistency.<n>Our analysis reveals a fundamental trade-off: perplexity methods suffer from substantial model error due to the absence of a proper consistency function.<n>We propose Reasoning-Pruning Perplexity Consistency (RPC), which integrates perplexity with self-consistency, and Reasoning Pruning, which eliminates low-probability reasoning paths.
  arXiv  Detail & Related papers  (2025-02-01T18:09:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.