SAFE: Stepwise Atomic Feedback for Error correction in Multi-hop Reasoning
- URL: http://arxiv.org/abs/2604.01993v1
- Date: Thu, 02 Apr 2026 12:59:30 GMT
- Title: SAFE: Stepwise Atomic Feedback for Error correction in Multi-hop Reasoning
- Abstract summary: We propose SAFE, a dynamic benchmarking framework that replaces the ungrounded Chain-of-Thought (CoT) with a strictly verifiable sequence of grounded entities.<n>Our framework operates across two phases: (1) train-time verification, where we establish an atomic error taxonomy and a Knowledge Graph (KG)-grounded verification pipeline, identifying up to 14% of instances as unanswerable, and (2) inference-time verification, where a feedback model trained on this verified dataset dynamically detects ungrounded steps in real-time.
- Score: 35.58279719577653
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Multi-hop QA benchmarks frequently reward Large Language Models (LLMs) for spurious correctness, masking ungrounded or flawed reasoning steps. To shift toward rigorous reasoning, we propose SAFE, a dynamic benchmarking framework that replaces the ungrounded Chain-of-Thought (CoT) with a strictly verifiable sequence of grounded entities. Our framework operates across two phases: (1) train-time verification, where we establish an atomic error taxonomy and a Knowledge Graph (KG)-grounded verification pipeline to eliminate noisy supervision in standard benchmarks, identifying up to 14% of instances as unanswerable, and (2) inference-time verification, where a feedback model trained on this verified dataset dynamically detects ungrounded steps in real-time. Experimental results demonstrate that SAFE not only exposes the critical flaws of existing benchmarks at train-time, but also significantly outperforms standard baselines, achieving an average accuracy gain of 8.4 pp while guaranteeing verifiable trajectories at inference-time.
Related papers
- Trust Your Guide Only When Certain: Uncertainty-Aware Sparse Alignment at Inference Time [16.36057592850375]
Weak supervisors exhibit pervasive high entropy across the vast majority of tokens.<n>Trust-based Uncertainty Sparse Alignment (TUSA) reframes alignment as a dynamic arbitration process.<n>Experiments show that TUSA consistently improves both safety alignment and general helpfulness.
arXiv Detail & Related papers (2026-09-01T03:05:49Z) - Verification-Aware Training for Speculative Decoding [57.84976863792784]
We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
arXiv Detail & Related papers (2026-08-31T01:42:10Z) - TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models [76.92890872338235]
Large-scale pre-trained Vision-Language models (VLMs) exhibit strong zero-shot generalization, yet are highly vulnerable to imperceptible adversarial perturbations.<n>To enhance robustness without requiring downstream task-specific retraining, we propose TAME, a novel test-time defense.
arXiv Detail & Related papers (2026-05-17T18:07:08Z) - SPREG: Structured Plan Repair with Entropy-Guided Test-Time Intervention for Large Language Model Reasoning [23.62252234067012]
Large Language Models (LLMs) are prone to logical hallucinations and drifts during long-chain reasoning.<n>We propose SPREG (Structured Plan-guided Real-time Entropy Gating), a lightweight inference-time framework for surgical error rectification.<n>Our experiments demonstrate significant gains, notably a 20.0% absolute accuracy improvement on AIME25.
arXiv Detail & Related papers (2026-04-20T06:55:26Z) - FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning [49.65751420291115]
Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps.<n>We propose FACT-E, a causality-inspired framework for evaluating CoT quality.<n>We show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learnings.
arXiv Detail & Related papers (2026-04-12T15:35:08Z) - LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection [11.420370070522205]
We introduce LiveFact, a benchmark that simulates the real-world "fog of war" in misinformation detection.<n>LiveFact uses dynamic, temporal evidence sets to evaluate models on their ability to reason with evolving, incomplete information.<n>Tests show that open-source Mixture-of-Experts models, such as Qwen3-235B-A22B, now match or outperform proprietary state-of-the-art systems.
arXiv Detail & Related papers (2026-04-06T16:20:47Z) - PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering [71.15346406323827]
We introduce PRIME, a benchmark for evaluating verifiers on Process-Outcome Alignment verification.<n>We find that current verifiers frequently fail to detect derivation flaws.<n>We propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME.
arXiv Detail & Related papers (2026-02-12T04:45:01Z) - From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs [58.02809208460186]
We revisit this paradox using high-quality traces from DeepSeek-R1 as demonstrations.<n>We find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal.<n>We introduce Insight-to-solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights.
arXiv Detail & Related papers (2025-09-27T08:59:31Z) - ASCoT: An Adaptive Self-Correction Chain-of-Thought Method for Late-Stage Fragility in LLMs [21.409155842171497]
Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs)<n>Errors introduced in the later stages of a CoT chain are significantly more likely to corrupt the final answer than identical errors made at the beginning.<n>We introduce the Adaptive Self-Correction Chain-of-Thought (ASCoT) method to address this specific vulnerability.
arXiv Detail & Related papers (2025-08-07T11:26:40Z) - Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance.
We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
arXiv Detail & Related papers (2024-02-13T11:25:20Z) - TRUST-LAPSE: An Explainable and Actionable Mistrust Scoring Framework
for Model Monitoring [4.262769931159288]
We propose TRUST-LAPSE, a "mistrust" scoring framework for continuous model monitoring.
We assess the trustworthiness of each input sample's model prediction using a sequence of latent-space embeddings.
Our latent-space mistrust scores achieve state-of-the-art results with AUROCs of 84.1 (vision), 73.9 (audio), and 77.1 (clinical EEGs)
arXiv Detail & Related papers (2022-07-22T18:32:38Z) - Higher Performance Visual Tracking with Dual-Modal Localization [106.91097443275035]
Visual Object Tracking (VOT) has synchronous needs for both robustness and accuracy.
We propose a dual-modal framework for target localization, consisting of robust localization suppressingors via ONR and the accurate localization attending to the target center precisely via OFC.
arXiv Detail & Related papers (2021-03-18T08:47:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.