From Chat to Interview: Agentic Requirements Elicitation with an Experience Ontology

• URL: http://arxiv.org/abs/2605.05828v1
• Date: Thu, 07 May 2026 08:03:03 GMT
• Title: From Chat to Interview: Agentic Requirements Elicitation with an Experience Ontology
• Abstract summary: This paper proposes an interview agent named OntoAgent for the elicitation of requirements guided by an experience ontology.<n>OntoAgent automatically analyzes domain-specific requirements descriptions to construct an experience ontology.<n>The results show that OntoAgent significantly outperforms existing baselines in both elicitation effectiveness and questioning efficiency.
• Score: 35.19488207612296
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Requirements elicitation interviews are crucial and time-consuming in requirements engineering, but heavily rely on the experience of requirements analysts. Although recent advancements in large language models (LLMs) have created new opportunities to automate this process, existing approaches rely solely on LLMs for free-form chat without taking into account the interview and development experience. That leads to the omission of implicit requirements and redundant questions. Practically, experienced analysts implicitly follow a structured cognitive framework when conducting requirements elicitation. Inspired by this observation, this paper proposes an interview agent named OntoAgent for the elicitation of requirements guided by an experience ontology. OntoAgent automatically analyzes domain-specific requirements descriptions to construct an experience ontology, which organizes requirements concerns into an ontology to support systematic and explainable interviews. During the interview, OntoAgent first performs four operations (i.e., ParseUser, ScoreOnto, ReRankOnto, GatePrune) guided by the ontology to identify the relevant requirement concerns. The selected concern is then combined with the current dialogue context to generate the elicitation question. To validate OntoAgent, we conduct comprehensive quantitative experiments using the widely adopted website application domain. The results show that OntoAgent significantly outperforms existing baselines in both elicitation effectiveness and questioning efficiency, achieving a 33% improvement in IRE and a 21% improvement in TKQR. Ablation studies further validate the contribution of each key design component. In addition, a qualitative user study demonstrates its practical advantages in real-world scenarios. We believe that OntoAgent can also be extended to requirements interview tasks in other domains.

Related papers

• StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [63.57092302641323]
  We introduce textbfStartupBench, an E2E agent benchmark grounded in market-validated AI startup products.<n>We study AI products with demonstrated adoption, together with their product and users, to identify real-world tasks for which AI has established practical demand.<n>Our results reveal that many market-validated agents remain beyond the reliable capabilities of current general-purpose agents.
  arXiv  Detail & Related papers  (2026-08-18T14:01:32Z)
• ReqElicitGym: An Evaluation Environment for Interview Competence in Conversational Requirements Elicitation [36.77382403204434]
  The bottleneck of automated software development is shifting from generating correct code to eliciting users' requirements.<n>Despite growing interest, the interview competence of LLMs in conversational requirements elicitation remains fully underexplored.<n>We propose ReqElicitGym, an interactive and automatic evaluation environment for assessing interview competence in conversational requirements elicitation.
  arXiv  Detail & Related papers  (2026-02-20T16:02:13Z)
• LLMREI: Automating Requirements Elicitation Interviews with LLMs [47.032121951473435]
  This study introduces LLMREI, a chat bot designed to conduct requirements elicitation interviews with minimal human intervention.<n>We evaluated its performance in 33 simulated stakeholder interviews.<n>Our findings indicate that LLMREI makes a similar number of errors compared to human interviewers, is capable of extracting a large portion of requirements, and demonstrates a notable ability to generate highly context-dependent questions.
  arXiv  Detail & Related papers  (2025-07-03T12:18:05Z)
• How to Elicit Explainability Requirements? A Comparison of Interviews, Focus Groups, and Surveys [2.153604655925499]
  This study examines the efficiency and effectiveness of three commonly used elicitation methods - focus groups, interviews, and online surveys.<n>The results show that interviews were the most efficient, capturing the highest number of distinct needs per participant per time spent.<n>We recommend a hybrid approach combining surveys and interviews to balance efficiency and coverage.
  arXiv  Detail & Related papers  (2025-05-29T17:23:14Z)
• EvalAgent: Discovering Implicit Evaluation Criteria from the Web [82.82096383262068]
  We introduce EvalAgent, a framework designed to automatically uncover nuanced and task-specific criteria.<n>EvalAgent mines expert-authored online guidance to propose diverse, long-tail evaluation criteria.<n>Our experiments demonstrate that the grounded criteria produced by EvalAgent are often implicit, yet specific.
  arXiv  Detail & Related papers  (2025-04-21T16:43:50Z)
• Using Large Language Models to Develop Requirements Elicitation Skills [1.1473376666000734]
  We propose conditioning a large language model to play the role of the client during a chat-based interview.<n>We find that both approaches provide sufficient information for participants to construct technically sound solutions.
  arXiv  Detail & Related papers  (2025-03-10T19:27:38Z)
• QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Search [89.97082652805904]
  We propose QLASS (Q-guided Language Agent Stepwise Search), to automatically generate annotations by estimating Q-values.<n>With the stepwise guidance, we propose a Q-guided generation strategy to enable language agents to better adapt to long-term value.<n>We empirically demonstrate that QLASS can lead to more effective decision making through qualitative analysis.
  arXiv  Detail & Related papers  (2025-02-04T18:58:31Z)
• RECOVER: Toward Requirements Generation from Stakeholders' Conversations [10.706772429994384]
  This paper introduces RECOVER, a novel conversational requirements engineering approach.<n>It supports practitioners in automatically extracting system requirements from stakeholder interactions.<n> Empirical evaluation shows promising performance, with generated requirements demonstrating satisfactory correctness, completeness, and actionability.
  arXiv  Detail & Related papers  (2024-11-29T08:52:40Z)
• Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice [59.705062519344]
  One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews.
  arXiv  Detail & Related papers  (2022-03-29T10:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.