VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement

• URL: http://arxiv.org/abs/2609.03153v1
• Date: Wed, 02 Sep 2026 20:36:45 GMT
• Title: VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement
• Abstract summary: We present an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations.<n>Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state.<n>We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time.
• Score: 57.86962208273888
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual fluency in generated video does not imply physical reliability, and a scalar quality score alone is incapable of indicating the obligation a clip violates or the moment it fails. We present VeriPhy, an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations and a statically validated execution plan before any frame is observed. During execution, observations gate and scope only declared calls to frozen low-level experts (e.g., segmentation and tracking, counting, eleven typed physical measurements over the resulting tracks, depth, OCR, and audio-event detection). Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state. Typed resolvers and fixed composition map usable records to a three-valued state (supported, contradicted, or unknown, surfaced as plausible, implausible, or abstain) with full provenance, so that every verdict is traceable to the evidence that produced it. We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time. On a 149-clip core carrying 304 such records, VeriPhy accounts for 228, against 164 for a published question-decomposition evaluator given the same clips and the same claims. Recall alone does not separate it from prompting the same backbone monolithically, which reaches 222; what separates them is that each decision retains its evidence record and provenance, making the traces auditable one verdict at a time and usable as the interface through which a critic verdict could be written back into generation.

Related papers

• Auditing the Synthetic Memoir: Measuring Scene-Level Confabulation in LLM-Generated Autobiography Against the Documented Record of the Life It Describes [0.0]
  We present the first quantified audit of a large language model's autobiography against a subject-specific ground-truth corpus.<n>A 366-day "page-a-day" book of first-person anecdotal entries was drafted with a conversational LLM.<n>Every day was audited at the anecdote-scene level against an independent verification corpus.
  arXiv  Detail & Related papers  (2026-08-23T20:10:50Z)
• From Traceability to Justifiability: Accountability Structures in Agentic Software Engineering [0.0]
  We measure, from public material only, whether AI records can express a claim and whether they hold where declared.<n>Across 188 double-graded cells we found no platform whose default record emits a content identity of a behavioral system.<n>An instrument computes realized assurance depth from a pipeline's published exhaust alone and compares it with the declared depth.
  arXiv  Detail & Related papers  (2026-08-21T16:45:00Z)
• Labels Are Not Endpoints: Treatment Leakage and Construct Validity in MCP Agent Security Evaluation [0.0]
  We audit a preserved campaign by tracing 10,200 execution rows to 180 model-bound requests, 45 semantic requests, and 15 observable stimuli.<n>We contribute a seven-link Integrity Chain and an executable, scope-bounded endpoint-integrity linter.
  arXiv  Detail & Related papers  (2026-08-13T06:44:40Z)
• The LLM Proposes, the Executive Disposes: A Self-Verifying Agent Instrument that Dissociates Commitment Drift from Binding Drift in Long-Horizon Agents [0.0]
  We present an agent instrument built so that verification is structural rather than post-hoc.<n>Every run invalidates itself when per-organ write-error, render-size, or salted-canary-echo floors are breached.<n>We report a clean, single-variable result on a failure every long-horizon agent suffers.
  arXiv  Detail & Related papers  (2026-08-04T15:10:37Z)
• Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI [39.17006317116773]
  We argue that "zero hallucination" is not a property a model possesses but a property a system enforces.<n>We present HALO, an assurance architecture which treats hallucination as a containable failure mode rather than an eliminable one.
  arXiv  Detail & Related papers  (2026-07-20T12:34:44Z)
• DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments [51.049999642138]
  We introduce DRNOISE, a 100-task benchmark for answer recovery under misleading evidence.<n>Each task has a unique gold answer supported by two corroborating indirect record chains; the paired noisy condition adds one plausible document that states a conflicting answer directly.<n>Across agents with strong clean-task performance, this single intervention causes 66-88 percentage-point accuracy drops.
  arXiv  Detail & Related papers  (2026-07-19T15:20:40Z)
• TRACE: An Operational Reasoning Schema for Auditable Agentic Commitments [0.0]
  TRACE (Typed Reasoning And Commitment Evidence) is a typed, versioned schema for recording reasoning traces.<n>The paper argues in three layers that reasoning is not in the language model.<n>A record-consumer contract states what a record guarantees and what a consumer must honor in return.
  arXiv  Detail & Related papers  (2026-07-14T08:08:21Z)
• WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning [63.9071746889252]
  Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages.<n>This source-internal evidence integration is central to real-world long-document analysis, yet existing benchmarks largely sidestep it.<n>We introduce WILDTRACE, a benchmark of 481 tasks over 214 naturally occurring long-form sources.
  arXiv  Detail & Related papers  (2026-07-10T12:09:21Z)
• Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting [44.70980205167796]
  AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs.<n>Recent efforts address isolated components but leave three gaps: they cover only narrow slices of the evaluation lifecycle and do not compose into a single interpretable record.<n>We present EvalCards, an operational reporting layer that composes benchmark metadata, evaluation run data, and model metadata into a unified record.
  arXiv  Detail & Related papers  (2026-06-08T17:55:02Z)
• ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence [57.37494162084001]
  Chain-of-Evidence (CoE) is a verifiability framework requiring every claim to be traceable to its evidence source.<n>CoE Audit is a post-hoc audit whose four integrity checks -- score verification, specification violation, reference verification, and method-code alignment -- apply uniformly to all systems.
  arXiv  Detail & Related papers  (2026-05-25T21:30:27Z)
• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.