Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation
- URL: http://arxiv.org/abs/2605.30000v2
- Date: Sun, 31 May 2026 12:00:02 GMT
- Title: Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation
- Abstract summary: textbfdataname is an 11-domain, 54-leaf, 1,000-query WebDev benchmark spanning both static-presentation and interactive-application tasks.<n>textbfframename, grounded in Flavell's metacognitive monitoring, separates evidence accumulation from judgment across three stages.
- Score: 24.920344869492066
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Front-end web code has become a core product surface for every frontier LLM release, yet evaluating these interactive applications at development speed remains costly because human-judged leaderboards like Arena do not scale. Existing automated proxies typically lean on reference implementations, test suites, or rigid checklists, and tend to miss the reasoned synthesis a human reviewer performs over a live session. We articulate a new evaluation regime that is simultaneously reference-free, autonomously driven, and holistically reasoned, and instantiate it through two artifacts. \textbf{\dataname} is an 11-domain, 54-leaf, 1,000-query WebDev benchmark spanning both static-presentation and interactive-application tasks, balanced across three difficulty tiers and three target-language groups, with briefs rewritten to resist recall from circulated prompts. \textbf{\framename}, grounded in Flavell's metacognitive monitoring, separates evidence accumulation from judgment across three stages: Static Perception forms a first impression from passive observation; Agent-Driven Interaction explores the application autonomously while capturing continuous screen video, audio, and per-step screenshots; Dynamic Scoring issues holistic functionality and aesthetics verdicts with structured failure attribution only after the evidence chain is complete. On \dataname, \framename aligns closely with expert human ratings while surfacing substantial headroom across 13 frontier LLMs on interactive web generation. \noindenthttps://anonymous.4open.science/r/Cookie-3CE/
Related papers
- BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes [54.96895469970198]
BrowserForge generates web interaction data at scale by driving many browser sandboxes in parallel over the open web.<n> BrowserForge couples three components: an open-web sourcing stage that exposes the agent to hundreds of thousands of real, openly reachable websites.<n>A rule-plus-model cleaning pipeline removes failed runs and rewrites the surviving reasoning into a single unified chain-of-thought style.
arXiv Detail & Related papers (2026-08-25T17:35:42Z) - DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation [8.452268647068166]
We introduce DashArena, to our knowledge the first benchmark for open-ended, task-grounded generation of interactive analytic dashboards.<n>DashArena requires each system to generate both a dashboard and a replayable interaction trajectory.<n>A browser replays the trajectory and turns the system's intended analytical workflow into reproducible visual and execution evidence.
arXiv Detail & Related papers (2026-08-11T06:54:37Z) - InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos [7.003662856458999]
We introduce textbfInteracVid, emphthe firstopen-source large-scale dataset that addresses this missing supervision<n>Everysample couples a preceding audio-visual context and an external stimulus with the realinteractive response that follows.<n>A ten-rater human study confirms that the extracted interactions are causal,natural, and temporally complete for both genuine and reconstructed queries.
arXiv Detail & Related papers (2026-08-02T11:28:13Z) - UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation [35.72992007209751]
We introduce the first benchmark targeting interaction inference, the ability to recover application behavior from screenshots alone.<n> UI2App comprises 327 screenshots grouped into 45 state-coherent screenshot sets for runnable multi-route web applications.<n>We design an end-to-end pipeline that evaluates each artifact along four dimensions: executability, navigation reachability, visual fidelity, and interaction inference.
arXiv Detail & Related papers (2026-07-07T14:08:55Z) - Rehearsed Multi-Agent Live Product Demonstrations with Real-Time Voice Question Answering [0.06524460254566902]
Rhetor is a multi-agent system that produces a rehearsed live demonstration with segment-synchronized narration and real-time voice question answering.<n>The architectural contributions are a cross-modal feature representation that merges UI exploration with source-code analysis.
arXiv Detail & Related papers (2026-06-29T13:36:51Z) - Interactive Tracking: A Human-in-the-Loop Paradigm with Memory-Augmented Adaptation [60.96010213186819]
We introduce Interactive Tracking, a new paradigm that allows users to guide the tracker at any time using natural language commands.<n>We present InteractTrack, the first large-scale benchmark for interactive tracking, containing 150 videos with dense bounding box annotations and timestamped language instructions.<n>Third, we introduce Interactive Memory-Augmented Tracking (IMAT), a new baseline that employs a dynamic memory mechanism to learn from user feedback and update tracking behavior accordingly.
arXiv Detail & Related papers (2026-04-02T12:33:27Z) - FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback [92.67587639164908]
We present FronTalk, a benchmark for front-end code generation with multi-modal feedback.<n>We focus on the front-end development task and curate FronTalk, a collection of 100 multi-turn dialogues.<n> Evaluation of 20 models reveals two key challenges that are under-explored systematically in the literature.
arXiv Detail & Related papers (2025-12-05T23:28:09Z) - IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video? [56.33950760097989]
IWR-Bench is a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video.<n>IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions.<n>This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code.
arXiv Detail & Related papers (2025-09-29T12:38:06Z) - OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding [50.72259772580637]
We introduce OST-Bench, a benchmark designed to evaluate Online Spatio-Temporal understanding from the perspective of an agent.<n>Built on an efficient data collection pipeline, OST-Bench consists of 1.4k scenes and 10k question-answer pairs collected from ScanNet, Matterport3D, and ARKitScenes.<n>We find that both complex-based spatial reasoning demands and long-term memory retrieval requirements significantly drop model performance along two separate axes.
arXiv Detail & Related papers (2025-07-10T17:56:07Z) - Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping [57.024913536420264]
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance on the design-to-code task.<n>We present the first systematic investigation of MLLMs in generating interactive webpages.
arXiv Detail & Related papers (2024-11-05T17:40:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.