A11YN: aligning LLMs for accessible web UI code generation

• URL: http://arxiv.org/abs/2510.13914v1
• Date: Wed, 15 Oct 2025 06:42:29 GMT
• Title: A11YN: aligning LLMs for accessible web UI code generation
• Abstract summary: We introduce A11yn, the first method that aligns code-generating large language models to reliably produce accessibility-compliant web UIs.<n>A11yn optimize a novel reward function that penalizes violations of the Web Content Accessibility Guidelines (WCAG)<n>For evaluation, we introduce RealUIReq-300, a benchmark of 300 real-world web UI requests grounded and manually curated from public web pages.
• Score: 23.43138311599752
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) have recently demonstrated strong capabilities in generating functional and aesthetic web interfaces directly from instructions. However, these models often replicate accessibility flaws from their training data, resulting in interfaces that exclude users with diverse needs and contexts. To address this gap, we introduce A11yn, the first method that aligns code-generating LLMs to reliably produce accessibility-compliant web UIs. A11yn optimizes a novel reward function that penalizes violations of the Web Content Accessibility Guidelines (WCAG), with penalties scaled to the severity of each violation as identified by an accessibility testing engine. To support training, we construct UIReq-6.8K, a dataset of 6,800 diverse instructions for web UI generation. For evaluation, we introduce RealUIReq-300, a benchmark of 300 real-world web UI requests grounded and manually curated from public web pages, spanning a broad range of use cases. Empirical results show that A11yn significantly outperforms strong baselines, lowering the Inaccessibility Rate by 60% over the base model while preserving semantic fidelity and visual quality of generated UIs. These findings demonstrate that accessibility can be systematically optimized within LLMs, showing the feasibility of aligning code generation for accessibility.

Related papers

• Learning to Detect UI Principle Violations via Reinforcement Learning [8.915333011185671]
  Small language models and coding agents increasingly generate web front-end code, yet their outputs are typically evaluated for functional correctness.<n>A generated interface may compile, render, and pass unit tests while still violating established interface quality principles.<n>We investigate whether a lightweight vision-language model can serve as an effective critic for generated interfaces.
  arXiv  Detail & Related papers  (2026-07-22T19:42:59Z)
• Reasoning for Mobile User Experience with Multimodal LLMs: Task, Benchmark, and Approach [14.935343848369486]
  We propose UI-UX, an MLLM based on Qwen3-VL-4B-Thinking foundation model and enhanced via reinforcement learning.<n>UX-UX achieves state-of-the-art (SOTA) performance on UXBench, attaining an accuracy of 0.7963.
  arXiv  Detail & Related papers  (2026-06-11T11:00:16Z)
• Enhanced Web Payload Classification Using WAMM: An AI-Based Framework for Dataset Refinement and Model Evaluation [0.0]
  WAMM is an AI-driven multiclass web attack detection framework designed to reveal the limitations of rule-based systems.<n>XGBoost reaches 99.59% accuracy with microsecond-level inference while deep learning models degrade under noisy augmentation.<n>These findings expose gaps in widely deployed rule-based defenses and demonstrate that curated training pipelines combined with efficient machine learning models enable a more resilient, real-time approach to web attack detection suitable for production WAF environments.
  arXiv  Detail & Related papers  (2025-12-29T17:10:36Z)
• WebRenderBench: Enhancing Web Interface Generation through Layout-Style Consistency and Reinforcement Learning [24.178675410636135]
  We present a large-scale benchmark of 45.1k webpages collected from real-world portal sites.<n>We also propose a novel evaluation metric that measures layout and style consistency from the final rendered pages.
  arXiv  Detail & Related papers  (2025-10-05T08:47:39Z)
• IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video? [56.33950760097989]
  IWR-Bench is a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video.<n>IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions.<n>This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code.
  arXiv  Detail & Related papers  (2025-09-29T12:38:06Z)
• ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search [53.40810298627443]
  ReGUIDE is a framework for web grounding that enables MLLMs to learn data efficiently through self-generated reasoning and spatial-aware criticism.<n>Our experiments demonstrate that ReGUIDE significantly advances web grounding performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-05-21T08:36:18Z)
• Leveraging Vision-Language Models for Visual Grounding and Analysis of Automotive UI [0.0]
  This work introduces a vision-language framework to facilitate the understanding of and interaction with automotive UIs.<n>To support research in this field, AutomotiveUI-Bench-4K, an open-source dataset comprising 998 images with 4,208 annotations, is also released.<n>A Molmo-7B-based model is fine-tuned using Low-Rank Adaptation (LoRa), incorporating generated reasoning along with visual grounding and evaluation capabilities.
  arXiv  Detail & Related papers  (2025-05-09T09:01:52Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks.<n>However, improvement is plateauing due to the exhaustion of readily available high-quality data.<n>We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining [67.87810796668981]
  Information-Sensitive Cropping (ISC) and Self-Refining Dual Learning (SRDL)<n>Iris achieves state-of-the-art performance across multiple benchmarks with only 850K GUI annotations.<n>These improvements translate to significant gains in both web and OS agent downstream tasks.
  arXiv  Detail & Related papers  (2024-12-13T18:40:10Z)
• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• EDGE: Enhanced Grounded GUI Understanding with Enriched Multi-Granularity Synthetic Data [15.801018643716437]
  This paper aims to enhance the GUI understanding and interacting capabilities of large vision-language models (LVLMs) through a data-driven approach. We propose EDGE, a general data synthesis framework that automatically generates large-scale, multi-granularity training data from webpages across the Web. Our approach significantly reduces the dependence on manual annotations, empowering researchers to harness the vast public resources available on the Web to advance their work.
  arXiv  Detail & Related papers  (2024-10-25T10:46:17Z)
• Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation [67.18144414660681]
  We propose a Fast-Slow Test-Time Adaptation (FSTTA) approach for online Vision-and-Language Navigation (VLN) Our method obtains impressive performance gains on four popular benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T07:47:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.