UXBench: Measuring the Actionability of LLM-Generated UX Critiques

• URL: http://arxiv.org/abs/2606.16262v1
• Date: Mon, 15 Jun 2026 06:08:39 GMT
• Title: UXBench: Measuring the Actionability of LLM-Generated UX Critiques
• Abstract summary: We introduce UXBench, a benchmark for evaluating large language models (LLMs) as interaction-grounded UX judges.<n> UXBench comprises local-first runnable web fixtures spanning ten product-surface families, paired with coverage-gated rubric exploration.<n>We evaluate eight frontier models under both an automated repair-lift protocol and a blind human validation study.
• Score: 58.14408744110387
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly deployed as UX judges that inspect interfaces, diagnose usability problems, and propose repairs. Yet no controlled benchmark measures whether the resulting critiques are reliable and actionable across heterogeneous product surfaces. We introduce UXBench, a benchmark for evaluating LLMs as interaction-grounded UX judges. UXBench comprises local-first runnable web fixtures spanning ten product-surface families, paired with coverage-gated browser exploration that forces models to collect interaction evidence before reporting. Each judge model produces a structured UX report over seven rubric dimensions; report quality is measured by whether a fixed downstream repair agent can improve the interface based on the critique. We evaluate eight frontier models under both an automated repair-lift protocol and a blind human validation study. Results show that UX judging is neither saturated nor one dimensional: models differ meaningfully in report actionability, exhibit distinct rubric-level repair signatures, vary in fixture-level reliability, and trade leadership across surface categories

Related papers

• The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean [2.822987192839875]
  Agent benchmarks are increasingly used to compare large language models (LLMs) and guide deployment decisions.<n>We identify a double measurement confound: execution-critical decisions are performed by a fixed scaffold instead of the model, while the scorer evaluates outputs using criteria that may not reflect task correctness.<n>We unify these issues within a measurement-theoretic framework that characterizes when benchmark scores can be interpreted as evidence of model capability.
  arXiv  Detail & Related papers  (2026-09-06T21:23:11Z)
• From Benchmark Performance to Tool Deployment: Human-in-the-Loop Anomaly Detection [28.67239775192971]
  We evaluate 19 unsupervised anomaly detection models on the BowTie dataset.<n>We observe that model performance is less stable than typically reported on standard benchmarks.<n>We deploy a unified human-in-the-loop framework for manufactured-part inspection.
  arXiv  Detail & Related papers  (2026-08-07T21:31:20Z)
• DualEval: Joint Model-Item Calibration for Unified LLM Evaluation [67.99119926012686]
  DualEval is a latent model-item calibration framework that represents models and evaluation items in a shared space.<n>We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries.
  arXiv  Detail & Related papers  (2026-06-24T22:40:46Z)
• Latent Performance Profiling of Large Language Models [47.009623327601226]
  We introduce Latent Performance Profiling (LPP) -- a framework that derives task-agnostic diagnostics from hidden activations and output distributions.<n>Unlike static accuracy scores, LPP provides stable, architecture-sensitive signatures across models of similar size.<n>We show that models with similar benchmark scores can exhibit contrasting latent profiles, such as differences in entropy or adaptability.
  arXiv  Detail & Related papers  (2026-05-28T14:41:26Z)
• VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence [26.0945130521806]
  Existing benchmarks assess either textual answer correctness or pixel-level localization in isolation.<n>We introduce VISTAQA, a benchmark for joint evaluation of free-form answer correctness and pixel-level evidence grounding in visual question answering.<n>We introduce GROVE, a metric that enforces joint correctness by combining textual accuracy and grounding quality via a per-sample geometric mean.
  arXiv  Detail & Related papers  (2026-05-20T03:44:06Z)
• IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation [85.56193980646981]
  We propose IF-RewardBench, a comprehensive meta-evaluation benchmark for instruction-following.<n>For each instruction, we construct a preference graph containing all pairwise preferences among multiple responses.<n>Experiments on IF-RewardBench reveal significant deficiencies in current judge models.
  arXiv  Detail & Related papers  (2026-03-05T02:21:17Z)
• HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam [63.84155758655084]
  Humanity's Last Exam (HLE) has become a widely used benchmark for evaluating frontier large language models.<n>We introduce HLE-Verified, a verified and revised version of HLE with a transparent verification protocol and fine-grained error taxonomy.<n>We evaluate seven state-of-the-art language models on HLE and HLE-Verified, observing an average absolute accuracy gain of 7--10 percentage points.
  arXiv  Detail & Related papers  (2026-02-15T02:50:15Z)
• TabReX : Tabular Referenceless eXplainable Evaluation [15.411207072791806]
  TabReX is a reference-less, property-driven framework for evaluating tables generated by large language models.<n>It computes interpretable, rubric-aware scores that quantify structural and factual fidelity.<n>To asses robustness, we introduce TabReX-Bench, a large-scale benchmark spanning six domains and twelve planner-driven perturbation types.
  arXiv  Detail & Related papers  (2025-12-17T19:20:20Z)
• DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLM [35.910677096654574]
  Document parsing aims to transform unstructured PDF images into semi-structured data, facilitating the digitization and utilization of information in diverse domains.<n>Common practice often selects the top-performing model on standard benchmarks.<n>We introduce DOCR-Inspector, which formalizes document parsing assessment as fine-grained error detection and analysis.
  arXiv  Detail & Related papers  (2025-12-11T13:16:33Z)
• Visual Document Understanding and Question Answering: A Multi-Agent Collaboration Framework with Test-Time Scaling [83.78874399606379]
  We propose MACT, a Multi-Agent Collaboration framework with Test-Time scaling.<n>It comprises four distinct small-scale agents, with clearly defined roles and effective collaboration.<n>It shows superior performance with a smaller parameter scale without sacrificing the ability of general and mathematical tasks.
  arXiv  Detail & Related papers  (2025-08-05T12:52:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.