LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents

• URL: http://arxiv.org/abs/2606.29399v1
• Date: Sun, 28 Jun 2026 13:45:59 GMT
• Title: LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents
• Abstract summary: We frame this task as planning: an agent observes evidence collected so far, picks next document fragment to inspect, and stops when the evidence is sufficient.<n>The agent operates over a vectorless document tree using browse, read, and search tools, and maintains a dynamic knowledge graph (KG) as state.<n>On a 200-question benchmark over NuScale Final Safety Analysis Report (FSAR) documents, the system reaches 81.5% accuracy with a fidelity of 0.93.
• Score: 2.3651168422805027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled across multiple chapters. We frame this task as planning: an LLM-based agent observes the evidence collected so far, picks the next document fragment to inspect, and stops when the evidence is sufficient. The agent operates over a vectorless document tree using browse, read, and search tools, and maintains a dynamic knowledge graph (KG) as state. On a 200-question benchmark over NuScale Final Safety Analysis Report (FSAR) documents, the system reaches 81.5% accuracy with a RAGAS Faithfulness of 0.93. The dominant performance factor is planning: against PageIndex, which uses the same document tree without state-conditioned action selection, the gap is +38.0pp (43.5% to 81.5%, p<0.001). The system also outperforms LightRAG (73.0%, p<0.05), HippoRAG (70.5%, p<0.01), and GraphRAG (49.5%, p<0.001), and matches RAPTOR (75.5%, p=0.11) without offline indexing. Edge inference adds 2.8x cost without raising accuracy; we retain it as a traceability module. Of 7,391 inferred edges, 3 Violates edges (0.04%) flag scope boundaries (Q058) and partial conformance (Q176) as typed annotations that a human reviewer can audit.

Related papers

• PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents [85.82624962221026]
  We introduce PARSER, which decouples reading from reasoning.<n>A bank of lightweight subagents each bound to a single chunk read the entire document in parallel.<n>On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline.
  arXiv  Detail & Related papers  (2026-09-06T16:19:01Z)
• From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation [3.807723380039104]
  Technical documentation is written for human developers.<n>An increasing share of software changes is now authored by autonomous coding agents.<n>We conduct a behaviour-grounded study of agent-documentation interaction across two public datasets.
  arXiv  Detail & Related papers  (2026-08-20T15:51:54Z)
• The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context [45.88028371034407]
  We report a controlled case study in code repair, not a new phenomenon.<n>We toggle one flag (one-chunk-perfile deduplication) on an otherwise identical stack.<n>We map where it holds: it reverses on a lexical BM25 (3.2pp, significant cross-paradigm interaction)<n>We show that at a tight fixed budget, do not hard-deduplicate by file, and A/B packing policies against the task, not the metric flag was tuned.
  arXiv  Detail & Related papers  (2026-08-14T19:22:50Z)
• Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations [0.0]
  Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query.<n>We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound.<n>We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations.
  arXiv  Detail & Related papers  (2026-08-06T17:23:13Z)
• DocAtlas: Long-Document Understanding as Mutable-State Interaction [67.90400271544678]
  We present DocAtlas, a system that treats long-document understanding as a mutable-state information-seeking process.<n>Given a document and a question, DocAtlas exposes search, reading, note-taking, and review tools, maintains a hierarchical tree and note store, and updates both as the agent records evidence.<n>With GPT-5.4, DocAtlas reaches 71.4% on MMLongBench-Doc, exceeding the human-expert reference of 65.8%.
  arXiv  Detail & Related papers  (2026-07-21T09:45:34Z)
• AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning [52.23991730630292]
  Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge.<n>We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer.<n>We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window.
  arXiv  Detail & Related papers  (2026-06-23T12:57:18Z)
• Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation [59.438696079345426]
  DICE is a training-free document-side strategy that splits documents into chunks, encodes them independently with a frozen model, and aggregates them back into a single vector.<n>On LongEmbed, DICE improves retrieval across four backbones, with the largest gains on slices beyond 4k tokens.<n>Across 12,779 filtered samples, DICE yields lower EDI than the single-vector baseline in 92.8% of cases.
  arXiv  Detail & Related papers  (2026-06-17T07:44:04Z)
• Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents [3.964533007623828]
  Modern large language model (LLM) agents need decision-relevant evidence at the moment of action.<n>We present the Counterfactual-Inspired Context Layer (CICL), which builds an instance context graph, estimates decision-oriented utility for candidate units, and compresses selected evidence into typed memory cards.<n>CICL provides a practical layer for measuring, ranking, and compressing decision-critical context for tool-using agents.
  arXiv  Detail & Related papers  (2026-06-06T13:02:28Z)
• WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts [46.85286238215375]
  Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence.<n>We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs.
  arXiv  Detail & Related papers  (2026-06-02T06:29:40Z)
• Self-Conditioned Positional HNSW for Overlap-Aware Retrieval in Chunked-Document RAG Systems: Method and Industrial Evidence-Quality Audit [0.0]
  Chunked-document retrieval is a common component of retrieval-augmented generation (RAG) systems.<n>We propose Self-Conditioned Positional HNSW, a lightweight modification that appends a low-dimensional positional code to chunk embeddings.
  arXiv  Detail & Related papers  (2026-06-01T01:42:55Z)
• Compliance-Scored Best-of-N Guardrail Orchestration for Multimodal Document Generation in Payments Dispute Defense [0.0]
  High-stakes enterprise document generation includes financial dispute narratives, compliance notices, and audit summaries.<n>Prior to a unified guardrail layer, production systems often stitched together separate PII redaction, content moderation, and format validation steps.<n>We present a guardrail orchestration layer for text and image inputs that couples multi-candidate generation with an explicit compliance score used for early exit.
  arXiv  Detail & Related papers  (2026-06-01T00:24:30Z)
• Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System [0.0]
  This study presents TorchSight, an open-source local system for security document classification built around a fine-tuned Qwen 3.5 27B model.<n>The model was trained on 78,358 samples from 13 permissively licensed sources and GPT-4 synthetic data covering seven security categories and 51 subcategories.<n>Results show that a fine-tuned local model can support accurate security document classification while keeping document processing under local control.
  arXiv  Detail & Related papers  (2026-05-19T18:18:19Z)
• How Far Is Document Parsing from Solved? PureDocBench: A Source-TraceableBenchmark across Clean, Degraded, and Real-World Settings [56.70440596502351]
  The past year has seen over 20 open document parsing models, yet benchmarks almost exclusively on OmniDocBench.<n>We present PureDocBench, a benchmark thatrenders document images from HTML/CSS, covering 10 domains, 66 pages, and 1,475 pages.
  arXiv  Detail & Related papers  (2026-05-08T09:30:31Z)
• OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning [34.493139441463995]
  We introduce OfficeQA Pro, a benchmark for evaluating AI agents on grounded, multi-document reasoning.<n>OfficeQA Pro consists of 133 questions that require precise document parsing, retrieval, and analytical reasoning.<n>We find that providing agents with a structured document representation produced by Databricks' ai_parse_document yields a 16.1% average relative performance gain.
  arXiv  Detail & Related papers  (2026-03-09T17:34:53Z)
• MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm [60.14048367611333]
  MonkeyOCR is a vision-language model for document parsing.<n>It advances the state of the art by leveraging a Structure-Recognition-Relation (SRR) triplet paradigm.
  arXiv  Detail & Related papers  (2025-06-05T16:34:57Z)
• LLMs Can Patch Up Missing Relevance Judgments in Evaluation [56.51461892988846]
  We use large language models (LLMs) to automatically label unjudged documents. We simulate scenarios with varying degrees of holes by randomly dropping relevant documents from the relevance judgment in TREC DL tracks. Our method achieves a Kendall tau correlation of 0.87 and 0.92 on an average for Vicuna-7B and GPT-3.5 Turbo respectively.
  arXiv  Detail & Related papers  (2024-05-08T00:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.