DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation

• URL: http://arxiv.org/abs/2608.10567v1
• Date: Tue, 11 Aug 2026 06:54:37 GMT
• Title: DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation
• Abstract summary: We introduce DashArena, to our knowledge the first benchmark for open-ended, task-grounded generation of interactive analytic dashboards.<n>DashArena requires each system to generate both a dashboard and a replayable interaction trajectory.<n>A browser replays the trajectory and turns the system's intended analytical workflow into reproducible visual and execution evidence.
• Score: 8.452268647068166
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Analytic dashboards combine coordinated views and interactions for data exploration and decision-making. Recent models can generate them from data and natural-language goals, but evaluating their usefulness remains difficult. Dashboard generation is open-ended, and neither static appearance nor successful execution alone captures analytical support and interaction quality. We introduce DashArena, to our knowledge the first benchmark for open-ended, task-grounded generation of interactive analytic dashboards. Its key innovation is to require each system to generate both a dashboard and a replayable interaction trajectory. A browser executor replays the trajectory and turns the system's intended analytical workflow into reproducible visual and execution evidence. A VLM judge compares candidates using this evidence, and Bradley--Terry aggregation produces the leaderboard. We further distill the judge into the open-weight DashJudge-8B. Human evaluations show that DashJudge-8B effectively reproduces human judgments and ablations show that interaction evidence improves judge agreement. Experiments with frontier models reveal persistent rendering, analytical, and interaction failures. Together, these results show that realistic dashboard generation remains challenging and that interaction-aware evaluation captures failures missed by static or execution-only checks.

Related papers

• Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis [55.78822230833247]
  We present Inter-X++, a comprehensive and large-scale benchmark designed to empower versatile HHI analysis.<n>Inter-X++ provides 11,388 high-fidelity interaction sequences and over 8.1M frames, featuring precise whole-body movements and detailed finger articulations.<n>We propose OpenHHI, a single and unified HHI representation and modeling framework that jointly optimize interaction reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2026-08-20T17:51:48Z)
• Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards [50.38481199112112]
  We introduce Dashboard2Code, a novel task that requires a model to proactively explore an interactive dashboard, acquire and integrate feedback from its own interactions, and generate code that reproduces the target dashboard.<n>We present DashboardMimic, the first Plotly+Dash benchmark for Dashboard2Code, comprising 180 carefully designed and manually verified dashboard-code pairs spanning three difficulty levels and covering eight common real-world interaction patterns.<n>We propose an automated evaluation framework tailored to dashboards that combines code semantic analysis with dynamic interaction-based testing to assess visual and interaction consistency.
  arXiv  Detail & Related papers  (2026-07-06T06:58:43Z)
• Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation [24.920344869492066]
  textbfdataname is an 11-domain, 54-leaf, 1,000-query WebDev benchmark spanning both static-presentation and interactive-application tasks.<n>textbfframename, grounded in Flavell's metacognitive monitoring, separates evidence accumulation from judgment across three stages.
  arXiv  Detail & Related papers  (2026-05-28T14:30:33Z)
• Interactive Tracking: A Human-in-the-Loop Paradigm with Memory-Augmented Adaptation [60.96010213186819]
  We introduce Interactive Tracking, a new paradigm that allows users to guide the tracker at any time using natural language commands.<n>We present InteractTrack, the first large-scale benchmark for interactive tracking, containing 150 videos with dense bounding box annotations and timestamped language instructions.<n>Third, we introduce Interactive Memory-Augmented Tracking (IMAT), a new baseline that employs a dynamic memory mechanism to learn from user feedback and update tracking behavior accordingly.
  arXiv  Detail & Related papers  (2026-04-02T12:33:27Z)
• DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards [44.69783955774917]
  DashboardQA is a benchmark designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards.<n>It includes 112 interactive dashboards from Tableau Public and 405 question-answer pairs with interactive dashboards spanning five categories: multiple-choice, factoid, hypothetical, multi-dashboard, and conversational.<n>Our findings indicate that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated.
  arXiv  Detail & Related papers  (2025-08-24T15:11:44Z)
• ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation [51.297873393639456]
  ArtifactsBench is a framework for automated visual code generation evaluation.<n>Our framework renders each generated artifact and captures its dynamic behavior through temporal screenshots.<n>We construct a new benchmark of 1,825 diverse tasks and evaluate over 30 leading Large Language Models.
  arXiv  Detail & Related papers  (2025-07-07T12:53:00Z)
• Interact with me: Joint Egocentric Forecasting of Intent to Interact, Attitude and Social Actions [25.464036307823974]
  SocialEgoNet is a graph-based framework that exploits task dependencies through a hierarchical learning approach.<n>SocialEgoNet uses body skeletons (keypoints from face, hands and body) extracted from only 1 second of video input for high inference speed.<n>For evaluation, we augment an existing egocentric human-agent interaction with new class labels and bounding box annotations.
  arXiv  Detail & Related papers  (2024-12-21T16:54:28Z)
• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases [18.434430375939755]
  As machine learning (ML) systems become increasingly widespread, it is necessary to audit these systems for biases prior to their deployment. Recent research has developed algorithms for effectively identifying intersectional bias in the form of interpretable, underperforming subsets (or slices) of the data. We propose Visual Auditor, an interactive visualization tool for auditing and summarizing model biases.
  arXiv  Detail & Related papers  (2022-06-25T02:48:27Z)
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection [101.56529337489417]
  We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of human, action, object> in images. We argue that multi-level consistencies among objects, actions and interactions are strong cues for generating semantic representations of rare or previously unseen HOIs. Our model takes visual features of candidate human-object pairs and word embeddings of HOI labels as inputs, maps them into visual-semantic joint embedding space and obtains detection results by measuring their similarities.
  arXiv  Detail & Related papers  (2020-08-14T09:11:18Z)
• A Graph-based Interactive Reasoning for Human-Object Interaction Detection [71.50535113279551]
  We present a novel graph-based interactive reasoning model called Interactive Graph (abbr. in-Graph) to infer HOIs. We construct a new framework to assemble in-Graph models for detecting HOIs, namely in-GraphNet. Our framework is end-to-end trainable and free from costly annotations like human pose.
  arXiv  Detail & Related papers  (2020-07-14T09:29:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.