LadderTeam: Dual-Agent Laddering Elicitation Framework

• URL: http://arxiv.org/abs/2608.17029v1
• Date: Mon, 17 Aug 2026 18:27:33 GMT
• Title: LadderTeam: Dual-Agent Laddering Elicitation Framework
• Abstract summary: We present textbfLadderTeam, an open, reproducible framework that automates UX wireframe interviews using a dual-agent Large Language Model architecture.<n>An active interviewer agent executes one of three probing strategies to elicit actionable software requirements from user feedback comments.<n>A concurrent background Judge agent evaluates probe-response pairs and triggers real-time guardrails to prevent topic drift.
• Score: 41.99844472131922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Eliciting detailed and actionable software requirements from end-users is a critical phase in the iterative development of a software product or application. To ensure the feedback collected is detailed and actionable, software teams can leverage the laddering interview technique. While effective for ensuring granular and actionable items from the software feedback, these interviews are subject to several limitations. They are traditionally a manual process associated with a time and financial burden, limiting scalability; interviewers must balance probing for depth while managing interviewee behavioral and cultural constraints. To address these limitations, we present \textbf{LadderTeam}, an open, reproducible framework that automates UX wireframe interviews using a dual-agent Large Language Model (LLM) architecture. An active interviewer agent executes one of three probing strategies (ACV, 5-Whys, and JTBD) to elicit actionable software requirements from usability feedback comments, while a concurrent background Judge agent evaluates probe-response pairs and triggers real-time guardrails to prevent topic drift. To rigorously evaluate LLM laddering without participant variance confounds, we introduce a controlled simulation methodology utilizing scripted ground-truth transcripts to isolate probe quality as the sole experimental variable. Across 216 interviews, \textbf{LadderTeam} achieved 99.1\% chain convergence and an 81.0\% ground-truth actionable response match (86.1\% reluctant personality, 75.9\% terse personality) with zero drift across all runs. All evaluation code, all transcripts, inputs, and a live demonstration platform will be open-sourced upon acceptance.

Related papers

• AI-assisted Script Management for Requirements Elicitation Interviews [0.4376619220604389]
  This paper presents an AI-assisted elicitation workflow that combines theory-guided script generation grounded in business goals with live support for topic coverage tracking and on-demand follow-up question generation.<n>Based on a rubric derived from elicitation best practices, the AI-generated scripts score higher than training-only scripts.
  arXiv  Detail & Related papers  (2026-08-03T03:22:37Z)
• StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows [15.552044829103373]
  We present StructureClaw, an artifact-centered workbench for structural engineering requests.<n>We also introduce StructureClaw-Bench, an executable benchmark of 150 controlled scenarios.<n>Across ten agent-model configurations, each evaluated on the same 50 standard cases, the average Success Rate rises from 56.8% with the generic-skill baseline to 88.6% with the full automatic workflow.
  arXiv  Detail & Related papers  (2026-07-16T12:13:41Z)
• PolyInterview: An LLM-based Platform for Immersive Mock Interview Practice with Comprehensive Multimodal Assessment [22.1874861957095]
  PolyInterview is an immersive mock interview practice with comprehensive multimodal assessment.<n>PolyInterview uses the target job description and CV to generate questions tailored to the role and candidate.<n>Four parallel evaluators produce 13 behavior-level features that are aggregated into 10 assessment aspects and two competency tracks.
  arXiv  Detail & Related papers  (2026-07-11T13:33:50Z)
• Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation Traces [32.850551109913845]
  Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs)<n>We show that this prompt-centric paradigm overlooks a structural vulnerability in stateful, function-calling environments.<n>We exploit this architectural flaw through SMT, a black-box attack framework based on Simulated Moderation Traces.
  arXiv  Detail & Related papers  (2026-07-01T06:08:07Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• InspectCoder: Dynamic Analysis-Enabled Self Repair through interactive LLM-Debugger Collaboration [71.18377595277018]
  Large Language Models (LLMs) frequently generate buggy code with complex logic errors that are challenging to diagnose.<n>We present InspectCoder, the first agentic program repair system that empowers LLMs to actively conduct dynamic analysis via interactive debugger control.
  arXiv  Detail & Related papers  (2025-10-21T06:26:29Z)
• Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol [83.83217247686402]
  Large Language Models (LLMs) have evolved from simple text generators into complex software systems that integrate retrieval augmentation, tool invocation, and multi-turn interactions.<n>Their inherent non-determinism, dynamism, and context dependence pose fundamental challenges for quality assurance.<n>This paper decomposes LLM applications into a three-layer architecture: textbftextitSystem Shell Layer, textbftextitPrompt Orchestration Layer, and textbftextitLLM Inference Core.
  arXiv  Detail & Related papers  (2025-08-28T13:00:28Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)
• Automated test generation to evaluate tool-augmented LLMs as conversational AI agents [0.27309692684728615]
  We present a test generation pipeline to evaluate conversational AI agents. Our framework uses LLMs to generate diverse tests grounded on user-defined procedures. Our results show that while tool-augmented LLMs perform well in single interactions, they often struggle to handle complete conversations.
  arXiv  Detail & Related papers  (2024-09-24T09:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.