Look Before You Prompt, and After: Scaffolding Human-AI Collaboration in Software Tutorial Creation

• URL: http://arxiv.org/abs/2609.05563v1
• Date: Thu, 03 Sep 2026 21:26:36 GMT
• Title: Look Before You Prompt, and After: Scaffolding Human-AI Collaboration in Software Tutorial Creation
• Abstract summary: dBlocks is a tool for creating software tutorials with model-based instructions (LLMs)<n> dBlocks reduces friction in verification, with writers verifying code as they drafted rather than deferring or skipping it.<n>Our work offers implications for tools that scaffold human-centered design in SE and shows how human-centered design can guide the development of LLM tools.
• Score: 3.4240285806053072
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With LLMs, creating software tutorials now involves steering the model's output and shaping it into a coherent, accurate learning resource, yet existing LLM tools offer writers little support for this work. By analyzing interviews with technical writers ($N=17$), we identify three requirements for how they assemble and structure multiple LLM responses, curate the context the model uses, and verify the generated content. We designed a tool called dBlocks with the following features: blocks to scope content, a context manager to edit context, and inline execution to verify code. Following a human-centered design method, we iteratively refined the design through a user study ($N=5$). In a within-subjects lab study ($N=16$) comparing dBlocks with participants' preferred workflows for LLM-assisted authoring, participants reported significantly higher confidence in the tutorials they produced with dBlocks. In addition, the tool reduced friction in verification, with writers verifying code as they drafted rather than deferring or skipping it, and helped them avoid searching long chat histories by scoping their work into blocks that kept each tutorial section and its LLM conversation together. More broadly, our work offers implications for tools that scaffold human-AI collaboration in SE workflows and shows how human-centered design can guide the development of LLM-integrated tools.

Related papers

• The Role of LLMs in Collaborative Software Design [9.27915132566963]
  Large Language Models (LLMs) shape collaborative group work in software engineering.<n>This study focuses on one such collaborative task, namely software design.
  arXiv  Detail & Related papers  (2026-04-10T09:00:33Z)
• Sketch: A Toolkit for Streamlining LLM Operations [51.33202045501429]
  Large language models (LLMs) have achieved remarkable success. The flexibility of their output format poses challenges in controlling and harnessing the model's outputs. We present Sketch, an innovative toolkit designed to streamline LLM operations across diverse fields.
  arXiv  Detail & Related papers  (2024-09-05T08:45:44Z)
• What You Need is What You Get: Theory of Mind for an LLM-Based Code Understanding Assistant [0.0]
  A growing number of tools have used Large Language Models (LLMs) to support developers' code understanding. In this study, we designed an LLM-based conversational assistant that provides a personalized interaction based on inferred user mental state. Our results provide insights for researchers and tool builders who want to create or improve LLM-based conversational assistants to support novices in code understanding.
  arXiv  Detail & Related papers  (2024-08-08T14:08:15Z)
• CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis [5.929552001093879]
  We generate the first exclusively LLM-generated collaborative stories dataset called CollabStory.<n>We generate over 32k stories using open-source instruction-tuned LLMs.<n>We extend their authorship-related tasks for multi-LLM settings and present baselines for LLM-LLM collaboration.
  arXiv  Detail & Related papers  (2024-06-18T14:35:12Z)
• Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents [56.822238860147024]
  Augmenting large language models with external tools has emerged as a promising approach to extend their utility.<n>Previous methods manually parse tool documentation and create in-context demonstrations, transforming tools into structured formats for LLMs to use in their step-by-step reasoning.<n>We propose AutoTools, a framework that enables LLMs to automate the tool-use workflow.
  arXiv  Detail & Related papers  (2024-05-26T11:40:58Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• SPROUT: an Interactive Authoring Tool for Generating Programming Tutorials with the Visualization of Large Language Models [19.885485760758783]
  The rapid development of large language models (LLMs) has revolutionized the efficiency of creating programming tutorials. We introduce a novel approach that breaks down the programming tutorial creation task into actionable steps. We then present SPROUT, an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process.
  arXiv  Detail & Related papers  (2023-12-04T10:46:52Z)
• Lessons from Building StackSpot AI: A Contextualized AI Coding Assistant [2.268415020650315]
  A new breed of tools, built atop Large Language Models, is emerging. These tools aim to mitigate drawbacks by employing techniques like fine-tuning or enriching user prompts with contextualized information.
  arXiv  Detail & Related papers  (2023-11-30T10:51:26Z)
• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models [74.22729793816451]
  Large Language Models (LLMs) have made significant progress in utilizing tools, but their ability is limited by API availability. We propose CREATOR, a novel framework that enables LLMs to create their own tools using documentation and code realization. We evaluate CREATOR on MATH and TabMWP benchmarks, respectively consisting of challenging math competition problems.
  arXiv  Detail & Related papers  (2023-05-23T17:51:52Z)
• Low-code LLM: Graphical User Interface over Large Language Models [115.08718239772107]
  This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability.
  arXiv  Detail & Related papers  (2023-04-17T09:27:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.