Learning to Detect UI Principle Violations via Reinforcement Learning

• URL: http://arxiv.org/abs/2607.20690v1
• Date: Wed, 22 Jul 2026 19:42:59 GMT
• Title: Learning to Detect UI Principle Violations via Reinforcement Learning
• Abstract summary: Small language models and coding agents increasingly generate web front-end code, yet their outputs are typically evaluated for functional correctness.<n>A generated interface may compile, render, and pass unit tests while still violating established interface quality principles.<n>We investigate whether a lightweight vision-language model can serve as an effective critic for generated interfaces.
• Score: 8.915333011185671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Small language models and coding agents increasingly generate web front-end code, yet their outputs are typically evaluated primarily for functional correctness. A generated interface may compile, render, and pass unit tests while still violating established interface quality principles, including accessibility barriers, deceptive design patterns, poor visual hierarchy, and excessive decision complexity. Existing auditing approaches face a trade-off between cost, coverage, and scalability: expert human review provides rich judgment but is slow and expensive; frontier vision-language models offer broader reasoning capabilities but remain costly to deploy at scale; and rule-based tools such as axe-core and Lighthouse are inexpensive but primarily capture mechanically checkable accessibility issues. We investigate whether a lightweight vision-language model can serve as an effective critic for generated interfaces. We unify 19 interface-quality principles from three complementary sources of HCI knowledge: WCAG 2.2 accessibility standards, deceptive design taxonomies, and established theories of perception, cognition, and interaction. To train this critic, we construct a verified dataset of approximately 10,000 generated web pages by synthetically injecting known violations into clean, LLM-generated Tailwind pages. Continued reinforcement learning on a 4B vision-language model improves micro-F1 from 36\% to 84\%, with 13 of 19 principles exceeding 80\% F1. The resulting critic can audit generated interfaces, filter low-quality interface training data, and provide a reward signal for design-aware code generation. We release our data-generation recipe and injection/verification prompts to support reproducible evaluation and future work on scalable interface-quality assessment.

Related papers

• Bridging the Interpretation Gap in Accessibility Testing: Empathetic and Legal-Aware Bug Report Generation via Large Language Models [2.9451141736604396]
  textscHEAR transforms raw accessibility bug reports into stakeholder-oriented narratives.<n>We evaluate the framework on real-world accessibility issues collected from four popular Android applications.
  arXiv  Detail & Related papers  (2026-03-25T01:27:25Z)
• CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing [51.56484100374058]
  CLAIRE is a hybrid end-to-end learning framework that integrates unsupervised deep representation learning with supervised classification for intelligent quality control in smart manufacturing systems.<n>It employs an optimized deep autoencoder to transform raw input into a compact latent space, effectively capturing the intrinsic data structure while suppressing irrelevant or noisy features.<n>The proposed framework highlights the potential of integrating explainable AI with feature-aware regularization for robust fault detection.
  arXiv  Detail & Related papers  (2026-03-06T15:11:58Z)
• Graphing the Truth: Structured Visualizations for Automated Hallucination Detection in LLMs [0.0]
  This paper introduces a framework that organizes proprietary knowledge and model-generated content into interactive visual knowledge graphs.<n>Users can diagnose inconsistencies, identify weak reasoning chains, and supply corrective feedback.
  arXiv  Detail & Related papers  (2025-11-29T23:09:15Z)
• A11YN: aligning LLMs for accessible web UI code generation [23.43138311599752]
  We introduce A11yn, the first method that aligns code-generating large language models to reliably produce accessibility-compliant web UIs.<n>A11yn optimize a novel reward function that penalizes violations of the Web Content Accessibility Guidelines (WCAG)<n>For evaluation, we introduce RealUIReq-300, a benchmark of 300 real-world web UI requests grounded and manually curated from public web pages.
  arXiv  Detail & Related papers  (2025-10-15T06:42:29Z)
• Attribution-Guided Decoding [24.52258081219335]
  We introduce Attribution-Guided Decoding (AGD), an interpretability-based decoding strategy.<n>Instead of directly manipulating model activations, AGD considers a set of high-probability output token candidates.<n>We demonstrate AGD's efficacy across three challenging domains.
  arXiv  Detail & Related papers  (2025-09-30T14:21:40Z)
• SOI is the Root of All Evil: Quantifying and Breaking Similar Object Interference in Single Object Tracking [25.076012214989433]
  We present the first systematic investigation and quantification of Similar Object Interference (SOI)<n> eliminating interference sources leads to substantial performance improvements (AUC gains up to 4.35) across all SOTA trackers.<n>We construct SOIBench-the first semantic cognitive guidance benchmark specifically targeting SOI challenges.
  arXiv  Detail & Related papers  (2025-08-13T06:12:43Z)
• ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation [51.297873393639456]
  ArtifactsBench is a framework for automated visual code generation evaluation.<n>Our framework renders each generated artifact and captures its dynamic behavior through temporal screenshots.<n>We construct a new benchmark of 1,825 diverse tasks and evaluate over 30 leading Large Language Models.
  arXiv  Detail & Related papers  (2025-07-07T12:53:00Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks.<n>However, improvement is plateauing due to the exhaustion of readily available high-quality data.<n>We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• ReLearn: Unlearning via Learning for Large Language Models [64.2802606302194]
  We propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning.<n>This framework introduces Knowledge Forgetting Rate (KFR) and Knowledge Retention Rate (KRR) to measure knowledge-level preservation.<n>Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality output.
  arXiv  Detail & Related papers  (2025-02-16T16:31:00Z)
• Self-Improvement in Language Models: The Sharpening Mechanism [70.9248553790022]
  We offer a new perspective on the capabilities of self-improvement through a lens we refer to as sharpening.<n>Motivated by the observation that language models are often better at verifying response quality than they are at generating correct responses, we formalize self-improvement as using the model itself as a verifier during post-training.<n>We analyze two natural families of self-improvement algorithms based on SFT and RLHF.
  arXiv  Detail & Related papers  (2024-12-02T20:24:17Z)
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [139.77117915309023]
  CRITIC allows large language models to validate and amend their own outputs in a manner similar to human interaction with tools. Comprehensive evaluations involving free-form question answering, mathematical program synthesis, and toxicity reduction demonstrate that CRITIC consistently enhances the performance of LLMs.
  arXiv  Detail & Related papers  (2023-05-19T15:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.