Related papers: WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts

WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts

URL: http://arxiv.org/abs/2606.03220v1
Date: Tue, 02 Jun 2026 06:29:40 GMT
Title: WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts
Abstract summary: Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence.<n>We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs.
Score: 46.85286238215375
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence and miss the requirement-induced states and transitions that determine whether a page works. We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs (ICGs) of observable states, user-intent transitions, and DOM/visual assertions for implementation-agnostic browser execution. WebRISE spans 442 tasks across five input modalities (Text, Markdown, Sketch, Image, Video), with 5,495 transitions and 5,271 requirement checks that separate user-stated functions from implicit product-level constraints. Across 14 MLLMs, even the strongest model reaches only 65.6% transition validity and 66.3% requirement coverage, and visual quality is no proxy for behavior (Qwen3.6-35B-A3B on Markdown: V=80.8 yet T=15.5). Video gives the strongest interaction signal (+10.6 pp implicit coverage over Text), while implicit constraints persist; defect injection shows ICG-based scoring detects state errors at 2-16x the rate of checkpoint-style evaluation.

Related papers

EvolveScaler: Synthesizing Information-Evolution Contexts via Executable State Machines and Natural-Language Rendering [57.87517047585447]
We introduce EvolveScaler, a code-driven framework that defines information evolution before rendering it as natural language.<n>We instantiate EvolveScaler with 117 task prototypes and 159 final-question operators across five difficulty levels spanning approximately 7 to 1,200 events per instance.<n>On the very_long tier, the strongest model reaches 59.3% avg@5, while six models score below 10%.
arXiv Detail & Related papers (2026-09-08T08:40:05Z)
EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants [10.510136510866099]
Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve.<n>We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios.<n>We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs.
arXiv Detail & Related papers (2026-08-29T17:54:02Z)
PULSE: An Executable Contract Language for Spatiotemporal Knowledge Graph Engineering [2.2262915480980063]
We present PULSE, an Object-Process-Methodology-inspired language that localizes four operational roles and their write effects in one typed runtime.<n>The implemented contract fixes evidence non-overwrite, branch isolation, grounded multisubject timers, guarded state change, and declaration-ranked event ordering over time and space.<n>GeoSPARQL, SOSA, and SHACL remain generated views; Lean 4 checks kernel analogues for positions, evidence, clocks, monitors, atomicity, and branch source retention.
arXiv Detail & Related papers (2026-07-26T17:44:05Z)
PInVerify: An Offline Embodied Benchmark for Active Instance Verification [8.403971471573607]
Embodied agents have made strong progress in navigating to target objects, but reaching the goal vicinity does not guarantee that the agent has found the correct instance.<n>We address this gap with Active Instance Verification (AIV), a task in which an agent actively selects viewpoints around a candidate object to decide whether it matches a fine-grained natural-language description.<n>We introduce PInVerify, an offline embodied benchmark for AIV: 3,000 evaluation episodes across 18 object categories, delivered as multi-view captures.
arXiv Detail & Related papers (2026-05-28T22:42:38Z)
ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents [59.626170560327274]
We present textbfClawForge, a generator-backed benchmark framework for executable command-line categories under state conflict.<n>We instantiate this framework as the ClawForge-Bench (17 scenarios, 6 ability categories)
arXiv Detail & Related papers (2026-05-13T21:34:08Z)
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules [4.124344125532972]
Monitoring complex industrial assets relies on engineer-authored symbolic rules that trigger based on sensor conditions.<n> translating rules into maintenance steps requires asset-specific knowledge gained through years of practice.<n>We investigate whether LLMs can serve as decision support for this rule-to-action step and introduce ours, a benchmark of 6,690 expert-validated multiple-choice questions.
arXiv Detail & Related papers (2026-05-09T02:17:39Z)
ARIAL: An Agentic Framework for Document VQA with Precise Answer Localization [1.580774794371876]
We present ARIAL, a framework that orchestrates specialized tools to achieve both precise answer extraction and reliable spatial grounding.<n>We evaluate ARIAL on four benchmarks (DocVQA, FUNSD, CORD, and SROIE) using both textual accuracy (ANLS) and spatial precision.<n>Our work demonstrates how agentic orchestration of specialized tools can simultaneously improve performance and interpretability.
arXiv Detail & Related papers (2025-11-22T21:09:28Z)
IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video? [56.33950760097989]
IWR-Bench is a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video.<n>IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions.<n>This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code.
arXiv Detail & Related papers (2025-09-29T12:38:06Z)
Benchmarking MLLM-based Web Understanding: Reasoning, Robustness and Safety [9.588706798051955]
Multimodal large language models (MLLMs) are increasingly positioned as AI collaborators for building complex web-related applications.<n>We introduce a comprehensive web understanding benchmark, named WebRSSBench, that jointly evaluates Reasoning, Robustness, and Safety across eight tasks.<n>The benchmark is constructed from 729 websites and contains 3799 question answer pairs that probe multi-step inference over page structure, text, widgets, and safety-critical interactions.
arXiv Detail & Related papers (2025-09-26T02:38:14Z)
SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
arXiv Detail & Related papers (2025-03-11T17:53:02Z)
Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping [57.024913536420264]
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance on the design-to-code task.<n>We present the first systematic investigation of MLLMs in generating interactive webpages.
arXiv Detail & Related papers (2024-11-05T17:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.