VibeJam: A User Study Platform for Web Development with Agents

• URL: http://arxiv.org/abs/2608.29889v1
• Date: Sun, 30 Aug 2026 16:37:51 GMT
• Title: VibeJam: A User Study Platform for Web Development with Agents
• Abstract summary: We release VibeJam, a browser-based user study platform for users to collaborate with AI agents to develop websites.<n> VibeJam enables agent customization and uses the open-source Aider agent by default, and to mirror downstream use, we add diff review, chat and plan modes, and live website previews.<n>In a pilot study with 55 released, game-based website creation tasks, five experienced AI programmers rate our system as fun, simple, and resembling commercial tools, while 13 junior students use VibeJam to make websites of higher quality than agents in the same task.
• Score: 54.292180786970476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Programming with AI is increasingly agentic, users prompt LLMs to directly edit their code and review the changes, with adoption growing especially for web development tasks. Despite this growth, most NLP work uses offline evaluation and lacks support for online studies, losing insights into how programmers truly use coding agents. We release VibeJam, a browser-based user study platform for users to collaborate with AI agents to develop websites. VibeJam enables agent customization and uses the open-source Aider agent by default, and to mirror downstream use, we add diff review, chat and plan modes, and live website previews. In a pilot study with 55 released, game-based website creation tasks, five experienced AI programmers rate our system as fun, simple, and resembling commercial tools, while 13 junior students use VibeJam to make websites of higher quality than agents in the same task. We open-source VibeJam to spur extensions and support studies on how coding agents can help users.

Related papers

• AI Agents for Web Testing: A Case Study in the Wild [20.669140680308494]
  We present WebProber, a prototype AI agent-based web testing framework.<n>Given a URL, WebProber autonomously explores the website, simulating real user interactions, identifying bugs and usability issues, and producing a human-readable report.<n>We evaluate WebProber through a case study of 120 academic personal websites, where it uncovered 29 usability issues--many of which were missed by traditional tools.
  arXiv  Detail & Related papers  (2025-09-05T15:57:16Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systems [31.113305753414913]
  AUTOGEN STUDIO is a no-code developer tool for rapidly prototyping multi-agent systems. It provides an intuitive drag-and-drop UI for agent specification, interactive evaluation, and a gallery of reusable agent components.
  arXiv  Detail & Related papers  (2024-08-09T03:27:37Z)
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents [109.8507367518992]
  We introduce OpenHands, a platform for the development of AI agents that interact with the world in similar ways to a human developer.<n>We describe how the platform allows for the implementation of new agents, safe interaction with sandboxed environments for code execution, and incorporation of evaluation benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T17:50:43Z)
• Experiential Co-Learning of Software-Developing Agents [83.34027623428096]
  Large language models (LLMs) have brought significant changes to various domains, especially in software development. We introduce Experiential Co-Learning, a novel LLM-agent learning framework. Experiments demonstrate that the framework enables agents to tackle unseen software-developing tasks more effectively.
  arXiv  Detail & Related papers  (2023-12-28T13:50:42Z)
• OpenAgents: An Open Platform for Language Agents in the Wild [71.16800991568677]
  We present OpenAgents, an open platform for using and hosting language agents in the wild of everyday life. We elucidate the challenges and opportunities, aspiring to set a foundation for future research and development of real-world language agents.
  arXiv  Detail & Related papers  (2023-10-16T17:54:53Z)
• Agents: An Open-source Framework for Autonomous Language Agents [98.91085725608917]
  We consider language agents as a promising direction towards artificial general intelligence. We release Agents, an open-source library with the goal of opening up these advances to a wider non-specialist audience.
  arXiv  Detail & Related papers  (2023-09-14T17:18:25Z)
• ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents [27.222054181839095]
  ADVISER is an open-source, multi-domain dialog system toolkit. It enables the development of multi-modal (incorporating speech, text and vision) conversational agents. The final Python-based implementation of our toolkit is flexible, easy to use, and easy to extend.
  arXiv  Detail & Related papers  (2020-05-04T18:27:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.