StepGap: A Hybrid NLI-LLM Checker for Step-Level Evidence-Gap Detectionin Multi-Hop Question Answering

• URL: http://arxiv.org/abs/2605.24733v1
• Date: Sat, 23 May 2026 20:57:19 GMT
• Title: StepGap: A Hybrid NLI-LLM Checker for Step-Level Evidence-Gap Detectionin Multi-Hop Question Answering
• Abstract summary: StepGap is a hybrid NLI-LLM decision tree that detects step-level evidence gaps in multi-hop QA.<n>Used as a typed GRPO process reward, StepGap improves Qwen2.5-7B-Instruct Exact Match from $32.1pm0.3$ to $35.4pm0.9$ across three seeds.
• Score: 28.730693625995215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present \textbf{StepGap}, a hybrid NLI-LLM decision tree that detects step-level evidence gaps in multi-hop QA and emits one of three typed labels: \textsc{Contradicted Claim} (CC), \textsc{Irrelevant Evidence} (IE), or \textsc{Missing Bridge} (MB), each tied to a concrete repair action. On 82 multi-hop questions (181 annotated steps, $κ{=}0.704$), StepGap reaches sF1$=$72.0, within the bootstrap confidence interval of an LLM-only baseline (70.1) but with a more decomposable structure: every StepGap stage \emph{hurts} F1 when removed, while three of four LLM-only removals \emph{improve} F1 -- a sign of \emph{competing-error cancellation}, where internal stages mask each other's errors. We further expose a \emph{Q-F1 trap}: question-level F1 is mechanically inflated by checkers that flag every step, making step-level F1 the necessary diagnostic. Used as a typed GRPO process reward, StepGap improves Qwen2.5-7B-Instruct Exact Match from $32.1{\pm}0.3$ to $35.4{\pm}0.9$ across three seeds, with the single-run comparison showing a $+5.6$ Avg EM gain over the matched Search-R1 GRPO reproduction.

Related papers

• Ockhamareto: Pareto-Gated Segment-Level Credit Assignment for Concise Unit-Test Generation with Reinforcement Learning [64.49364696628294]
  We introduce textbfOckhamareto, a single-shot GRPO framework for unit-test generation and selection.<n>Ockhamareto has two principal components: (i)a emphPareto-gated Bonus that rewards only rollouts non-dominated in(mutation, $-$#tests) space, and (ii)emphToken-level Segment Credit, which attributes each test's marginal mutation kills back to the tokens of its unit-test block.
  arXiv  Detail & Related papers  (2026-08-25T12:20:02Z)
• An Interactive Agent for Requirement-Driven Candidate Sourcing [75.63902836826038]
  We present sys, the first requirements-driven candidate-sourcing agent.<n>It elicits, validates, retrieves, and verifies a vague people-request into a justified slate.<n>It dominates breadth ($100%$ coverage at $2.5times$ the yield) and is emphnear-orthogonal to the field.
  arXiv  Detail & Related papers  (2026-08-24T17:04:16Z)
• Full-Key Recovery and Forgery from One MQOM v2.1 Signature [0.0]
  We give a full-key-recovery attack on MQOM v2.1, a Round-3 candidate in the NIST additional-signature process.<n>We recover the complete signing key from one accepted signature and use it to sign a fresh message.
  arXiv  Detail & Related papers  (2026-08-10T15:04:50Z)
• Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes [70.61868608402723]
  We introduce $textbfSelf-Correcting Coupled Markov Jump Processes (SC-CMJP)<n>In conjunction with SC-CMJP, we introduce $textttCO_texttt2textttJump$, a novel training-free single-pass sampler for joint multimodal geneneration.<n>For training and evaluation purposes, we have created and will release three large-scale joint multimodal generation corpora.
  arXiv  Detail & Related papers  (2026-07-14T18:39:29Z)
• SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization [0.21485350418225238]
  Fine-tuned encoders deployed across heterogeneous NLP tasks face three compounding problems: mismatched inductive biases, class-imbalance corruption of feature statistics, and no mechanism to condition attention on external lexical knowledge.<n>We introduce textbfsurgellm, a unified transformer framework that addresses each with a dedicated lightweight module.
  arXiv  Detail & Related papers  (2026-06-23T07:47:21Z)
• Isolating LLM Alignment from Regex: Zero Coverage and Metric-Dependent Divergence Under Adversarial Mutation [51.56484100374058]
  Previous work found no measurable coverage gain from adding a live Gemini backend behind an active filter.<n>We introduce $L_5$-no-regex -- identical to $L_4$-real (Gemini-2.5-flash, token-budget cap, rate limit, output scrub) but with the nine-pattern filter disabled.<n>We evaluate it against adversarial probes across three sub-languages.
  arXiv  Detail & Related papers  (2026-06-12T18:54:24Z)
• The Security Budget of Code-LLM Prompt Hardening: Provable Limits Under Pass-Only Acceptance [0.0]
  We operationalize the floor as the emphTri-Audit Protocol, a two-axis reporting protocol that separates a prompt-side deductive registry attribute from a model-side empirical corollary.<n>A constrained best-of-family search over deterministic and guarded learned filters on CodeLlama-7B, Qwen2.5-Coder-7B/1.5B and DeepSeek-Coder-6.7B at $n=164$ yields the emphCross-Model Tri-Audit Invariance: of twenty-eight pass-preserving rows, twelve-changed-of-record learned-can
  arXiv  Detail & Related papers  (2026-06-02T08:22:14Z)
• SkiP: When to Skip and When to Refine for Efficient Robot Manipulation [50.03201984643502]
  textbfSkip Policy (SkiP) dynamically leaps over skip segments and intensively refines actions in key segments.<n>emphMotion Spectrum Keying (MSK) is a fast, task-agnostic procedure that detects local motion complexity from action signals.<n>Experiments show SkiP reduces executed steps by $15$--$40%$ while matching or improving success rates across various policy backbones.
  arXiv  Detail & Related papers  (2026-05-15T02:16:34Z)
• Mining Subscenario Refactoring Opportunities in Behaviour-Driven Software Test Suites: ML Classifiers and LLM-Judge Baselines [1.9537983097153042]
  Behaviour-Driven Development (BDD) software test suites accumulate duplicated step subsequences.<n>Three published patterns are available (within-file Background, within-repo reusable-scenario invocation, cross-organisational shared higher-level step)<n>No prior work automates which recurring subsequences are worth extracting or which mechanism applies.
  arXiv  Detail & Related papers  (2026-05-14T08:38:04Z)
• AAC: Admissible-by-Architecture Differentiable Landmark Compression for ALT [1.2891210250935148]
  AAC is a differentiable landmark-election module for ALT (A*, Landmarks, and Triangle) shortest-paths.<n>It is the first differentiable instance of the compress-while-admisibility tradition in classical search.
  arXiv  Detail & Related papers  (2026-04-22T16:31:21Z)
• Reducing Maintenance Burden in Behaviour-Driven Development: A Paraphrase-Robust Duplicate-Step Detector with a 1.1M-Step Open Benchmark [1.9537983097153042]
  Behaviour-Driven Development suites accumulate step-duplicate duplication with documented maintenance cost.<n>We release the largest cross-organisational BDD step corpus to date.
  arXiv  Detail & Related papers  (2026-04-22T11:44:05Z)
• Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols [51.56484100374058]
  We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
  arXiv  Detail & Related papers  (2026-04-20T13:25:40Z)
• Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing [2.6382975801439836]
  LLMs edit text and code by autoregressively regenerating the full output, even when most tokens appear verbatim in the input.<n>We study Copy-as-Decode, a decoding-layer mechanism that recasts edit generation as structured decoding over a two-primitive grammar.
  arXiv  Detail & Related papers  (2026-04-20T12:29:53Z)
• PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering [57.89576196160413]
  Large language models (LLMs) remain brittle on multi-hop question answering (MHQA)<n>We propose textbfPlanned Active Retrieval and Reasoning RAG (PAR$2$-RAG), a two-stage framework that separates emphcoverage from emphcommitment.
  arXiv  Detail & Related papers  (2026-03-30T23:52:54Z)
• SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents [52.20768003832476]
  We analyze execution traces on $$-Bench (Airline/Retail) and SWE-Bench Verified.<n>We formalize emphdecisive deviations, earliest action, level divergences that flip success to failure.<n>We introduce cm, a model-agnostic, gradient-free, test-time safeguard.
  arXiv  Detail & Related papers  (2025-11-26T01:28:22Z)
• Few-shot Reranking for Multi-hop QA via Language Model Prompting [56.454088569241534]
  We study few-shot reranking for multi-hop QA with open-domain questions. We propose PromptRank, which relies on large language models prompting for multi-hop path reranking. PromptRank yields strong retrieval performance on HotpotQA with only 128 training examples.
  arXiv  Detail & Related papers  (2022-05-25T10:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.