UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents

• URL: http://arxiv.org/abs/2504.09407v2
• Date: Mon, 21 Apr 2025 05:22:55 GMT
• Title: UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents
• Authors: Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang,
• Abstract summary: Recent advances in Large Language Model-simulated Agent (textbfLLM Agent) research inspired us to design textbfUXAgent.<n>Our system features a Persona Generator module, an LLM Agent module, and a Universal Browser Connector module to automatically generate thousands of simulated users.
• Score: 33.901185088456614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Usability testing is a fundamental research method that user experience (UX) researchers use to evaluate and iterate a web design, but\textbf{ how to evaluate and iterate the usability testing study design } itself? Recent advances in Large Language Model-simulated Agent (\textbf{LLM Agent}) research inspired us to design \textbf{UXAgent} to support UX researchers in evaluating and reiterating their usability testing study design before they conduct the real human-subject study. Our system features a Persona Generator module, an LLM Agent module, and a Universal Browser Connector module to automatically generate thousands of simulated users to interactively test the target website. The system also provides an Agent Interview Interface and a Video Replay Interface so that the UX researchers can easily review and analyze the generated qualitative and quantitative log data. Through a heuristic evaluation, five UX researcher participants praised the innovation of our system but also expressed concerns about the future of LLM Agent usage in UX studies.

Related papers

• UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design [33.901185088456614]
  Recent advances in Large Language Model-simulated Agent (LLM-Agent) research inspired us to design UXAgent.<n>Our system features an LLM-Agent module and a universal browser connector module so that UX researchers can automatically generate thousands of simulated users to test the target website.
  arXiv  Detail & Related papers  (2025-02-18T05:55:18Z)
• Commit0: Library Generation from Scratch [77.38414688148006]
  Commit0 is a benchmark that challenges AI agents to write libraries from scratch. Agents are provided with a specification document outlining the library's API as well as a suite of interactive unit tests. Commit0 also offers an interactive environment where models receive static analysis and execution feedback on the code they generate.
  arXiv  Detail & Related papers  (2024-12-02T18:11:30Z)
• GUI Agents with Foundation Models: A Comprehensive Survey [91.97447457550703]
  This survey consolidates recent research on (M)LLM-based GUI agents.<n>We identify key challenges and propose future research directions.<n>We hope this survey will inspire further advancements in the field of (M)LLM-based GUI agents.
  arXiv  Detail & Related papers  (2024-11-07T17:28:10Z)
• AgentSquare: Automatic LLM Agent Search in Modular Design Space [16.659969168343082]
  Large Language Models (LLMs) have led to a rapid growth of agentic systems capable of handling a wide range of complex tasks. We introduce a new research problem: Modularized LLM Agent Search (MoLAS)
  arXiv  Detail & Related papers  (2024-10-08T15:52:42Z)
• Automatic benchmarking of large multimodal models via iterative experiment programming [71.78089106671581]
  We present APEx, the first framework for automatic benchmarking of LMMs. Given a research question expressed in natural language, APEx leverages a large language model (LLM) and a library of pre-specified tools to generate a set of experiments for the model at hand. The report drives the testing procedure: based on the current status of the investigation, APEx chooses which experiments to perform and whether the results are sufficient to draw conclusions.
  arXiv  Detail & Related papers  (2024-06-18T06:43:46Z)
• System for systematic literature review using multiple AI agents: Concept and an empirical evaluation [5.194208843843004]
  We introduce a novel multi-AI agent model designed to fully automate the process of conducting Systematic Literature Reviews. The model operates through a user-friendly interface where researchers input their topic. It generates a search string used to retrieve relevant academic papers. The model then autonomously summarizes the abstracts of these papers.
  arXiv  Detail & Related papers  (2024-03-13T10:27:52Z)
• BASES: Large-scale Web Search User Simulation with Large Language Model based Agents [108.97507653131917]
  BASES is a novel user simulation framework with large language models (LLMs) Our simulation framework can generate unique user profiles at scale, which subsequently leads to diverse search behaviors. WARRIORS is a new large-scale dataset encompassing web search user behaviors, including both Chinese and English versions.
  arXiv  Detail & Related papers  (2024-02-27T13:44:09Z)
• MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization [86.61052121715689]
  MatPlotAgent is a model-agnostic framework designed to automate scientific data visualization tasks. MatPlotBench is a high-quality benchmark consisting of 100 human-verified test cases.
  arXiv  Detail & Related papers  (2024-02-18T04:28:28Z)
• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation [96.71370747681078]
  We introduce MLAgentBench, a suite of 13 tasks ranging from improving model performance on CIFAR-10 to recent research problems like BabyLM. For each task, an agent can perform actions like reading/writing files, executing code, and inspecting outputs. We benchmark agents based on Claude v1.0, Claude v2.1, Claude v3 Opus, GPT-4, GPT-4-turbo, Gemini-Pro, and Mixtral and find that a Claude v3 Opus agent is the best in terms of success rate.
  arXiv  Detail & Related papers  (2023-10-05T04:06:12Z)
• Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applications [4.139846693958609]
  This paper proposes a new methodology for integrating generative UX research into developing domain NLP applications. Generative UX research employs domain users at the initial stages of prototype development, i.e., ideation and concept evaluation, and the last stage for evaluating system usefulness and user utility.
  arXiv  Detail & Related papers  (2023-06-28T12:17:45Z)
• Agents for Automated User Experience Testing [4.6453787256723365]
  We propose an agent based approach for automatic UX testing. We develop agents with basic problem solving skills and a core affect model. Although this research is still at a primordial state, we believe the results here make a strong case for the use of intelligent agents.
  arXiv  Detail & Related papers  (2021-04-13T14:13:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.