Qualitative Evaluation of LLM-Designed GUI

• URL: http://arxiv.org/abs/2601.22759v1
• Date: Fri, 30 Jan 2026 09:33:44 GMT
• Title: Qualitative Evaluation of LLM-Designed GUI
• Abstract summary: Large Language Models (LLMs) are being explored for automated graphical user interface (GUI) design.<n>This study investigates the usability and adaptability of LLM-generated interfaces by analysing their ability to meet diverse user needs.
• Score: 35.39124330470992
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As generative artificial intelligence advances, Large Language Models (LLMs) are being explored for automated graphical user interface (GUI) design. This study investigates the usability and adaptability of LLM-generated interfaces by analysing their ability to meet diverse user needs. The experiments included utilization of three state-of-the-art models from January 2025 (OpenAI GPT o3-mini-high, DeepSeek R1, and Anthropic Claude 3.5 Sonnet) generating mockups for three interface types: a chat system, a technical team panel, and a manager dashboard. Expert evaluations revealed that while LLMs are effective at creating structured layouts, they face challenges in meeting accessibility standards and providing interactive functionality. Further testing showed that LLMs could partially tailor interfaces for different user personas but lacked deeper contextual understanding. The results suggest that while LLMs are promising tools for early-stage UI prototyping, human intervention remains critical to ensure usability, accessibility, and user satisfaction.

Related papers

• MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces [97.62557395494962]
  We use crowdsourcing to benchmark GPT-4o, Claude, and Llama across 30 interfaces.<n>Our results show that MLLMs approximate human preferences on some dimensions but diverge on others.
  arXiv  Detail & Related papers  (2025-10-09T20:00:41Z)
• Generative Interfaces for Language Models [70.25765232527762]
  We propose a paradigm in which large language models (LLMs) respond to user queries by proactively generating user interfaces (UIs)<n>Our framework leverages structured interface-specific representations and iterative refinements to translate user queries into task-specific UIs.<n>Results show that generative interfaces consistently outperform conversational ones, with up to a 72% improvement in human preference.
  arXiv  Detail & Related papers  (2025-08-26T17:43:20Z)
• GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking [55.762798168494726]
  GUI-ReRank is a novel framework that integrates rapid embedding-based constrained retrieval models with highly effective MLLM-based reranking techniques.<n>We evaluated our approach on an established NL-based GUI retrieval benchmark.
  arXiv  Detail & Related papers  (2025-08-05T10:17:38Z)
• GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototyping [55.762798168494726]
  Large Language Models (LLMs) with their impressive code generation capabilities offer a promising approach for automating GUI prototyping.<n>But there is a gap between current LLM-based prototyping solutions and traditional user-based GUI prototyping approaches.<n>We propose GUIDE, a novel LLM-driven GUI generation decomposition approach seamlessly integrated into the popular prototyping framework Figma.
  arXiv  Detail & Related papers  (2025-02-28T14:03:53Z)
• Zero-Shot Prompting Approaches for LLM-based Graphical User Interface Generation [53.1000575179389]
  We propose a Retrieval-Augmented GUI Generation (RAGG) approach, integrated with an LLM-based GUI retrieval re-ranking and filtering mechanism.<n>In addition, we adapt Prompt Decomposition (PDGG) and Self-Critique (SCGG) for GUI generation.<n>Our evaluation, which encompasses over 3,000 GUI annotations from over 100 crowd-workers with UI/UX experience, shows that SCGG, in contrast to PDGG and RAGG, can lead to more effective GUI generation.
  arXiv  Detail & Related papers  (2024-12-15T22:17:30Z)
• Large Language Model-Brained GUI Agents: A Survey [42.82362907348966]
  multimodal models have ushered in a new era of GUI automation.<n>They have demonstrated exceptional capabilities in natural language understanding, code generation, and visual processing.<n>These agents represent a paradigm shift, enabling users to perform intricate, multi-step tasks through simple conversational commands.
  arXiv  Detail & Related papers  (2024-11-27T12:13:39Z)
• Large Language User Interfaces: Voice Interactive User Interfaces powered by LLMs [5.06113628525842]
  We present a framework that can serve as an intermediary between a user and their user interface (UI) We employ a system that stands upon textual semantic mappings of UI components, in the form of annotations. Our engine can classify the most appropriate application, extract relevant parameters, and subsequently execute precise predictions of the user's expected actions.
  arXiv  Detail & Related papers  (2024-02-07T21:08:49Z)
• UI Layout Generation with LLMs Guided by UI Grammar [13.172638190095395]
  Large Language Models (LLMs) have stimulated interest among researchers and industry professionals. This paper proposes the introduction of UI grammar -- a novel approach to represent the hierarchical structure inherent in UI screens. The aim of this approach is to guide the generative capacities of LLMs more effectively and improve the explainability and controllability of the process.
  arXiv  Detail & Related papers  (2023-10-24T02:00:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.