Applying a Requirements-Focused Agile Management Approach for Machine Learning-Enabled Systems

• URL: http://arxiv.org/abs/2602.05042v1
• Date: Wed, 04 Feb 2026 20:49:02 GMT
• Title: Applying a Requirements-Focused Agile Management Approach for Machine Learning-Enabled Systems
• Authors: Lucas Romao, Luiz Xavier, Júlia Condé Araújo, Marina Condé Araújo, Ariane Rodrigues, Marcos Kalinowski,
• Abstract summary: Machine Learning (ML)-enabled systems challenge traditional Requirements Engineering (RE) and agile management.<n>Existing RE and agile practices remain poorly integrated and insufficiently tailored to these characteristics.<n>This paper reports on the practical experience of applying RefineML, a requirements-focused approach for the continuous and agile refinement of ML-enabled systems.
• Score: 1.3704574906282525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine Learning (ML)-enabled systems challenge traditional Requirements Engineering (RE) and agile management due to data dependence, experimentation, and uncertain model behavior. Existing RE and agile practices remain poorly integrated and insufficiently tailored to these characteristics. This paper reports on the practical experience of applying RefineML, a requirements-focused approach for the continuous and agile refinement of ML-enabled systems, which integrates ML-tailored specification and agile management approaches with best practices derived from a systematic mapping study. The application context concerns an industry-academia collaboration project between PUC-Rio and EXA, a Brazilian cybersecurity company. For evaluation purposes, we applied questionnaires assessing RefineML's suitability and overall acceptance and semi-structured interviews. We applied thematic analysis to the collected qualitative data. Regarding suitability and acceptance, the results of the questionnaires indicated high perceived usefulness and intention to use. Based on the interviews, stakeholders perceived RefineML as improving communication and facilitating early feasibility assessments, as well as enabling dual-track governance of ML and software work, allowing continuous refinement of the model while evolving the overall software project. However, some limitations remain, particularly related to difficulties in operationalizing ML concerns into agile requirements and in estimating ML effort.

Related papers

• Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMs [17.373348983049176]
  We propose RAGcceptance M2RE, a novel approach to generate acceptance criteria from multi-modal requirements data.<n>We show that our approach effectively reduces manual effort, captures nuanced stakeholder intent, and provides valuable criteria.<n>This research underscores the potential of multi-modal RAG techniques in streamlining software validation processes and improving development efficiency.
  arXiv  Detail & Related papers  (2025-08-09T08:35:40Z)
• Define-ML: An Approach to Ideate Machine Learning-Enabled Systems [1.3541839896498067]
  Machine learning (ML) in software systems demands specialized ideation approaches.<n>Traditional ideation methods like Lean Inception lack structured support for ML considerations.<n>This paper presents Define-ML, a framework that extends Lean Inception with tailored activities.
  arXiv  Detail & Related papers  (2025-06-25T17:11:26Z)
• A Practical Guide for Evaluating LLMs and LLM-Reliant Systems [1.1715858161748576]
  We present a practical evaluation framework which outlines how to proactively curate representative datasets and select meaningful evaluation metrics.<n>We employ meaningful evaluation methodologies that integrate well with practical development and deployment of systems that must adhere to real-world requirements and meet user-facing needs.
  arXiv  Detail & Related papers  (2025-06-16T01:18:16Z)
• OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics [82.0813150432867]
  We introduce OpenUnlearning, a standardized framework for benchmarking large language models (LLMs) unlearning methods and metrics.<n>OpenUnlearning integrates 13 unlearning algorithms and 16 diverse evaluations across 3 leading benchmarks.<n>We also benchmark diverse unlearning methods and provide a comparative analysis against an extensive evaluation suite.
  arXiv  Detail & Related papers  (2025-06-14T20:16:37Z)
• EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models [64.70546873396624]
  We present the Extremely Complex Instruction Following Benchmark (EIFBENCH) for evaluating large language models (LLMs)<n>EIFBENCH includes multi-task scenarios that enable comprehensive assessment across diverse task types concurrently.<n>We also propose the Segment Policy Optimization (SegPO) algorithm to enhance the LLM's ability to accurately fulfill multi-task workflow.
  arXiv  Detail & Related papers  (2025-06-10T02:39:55Z)
• Federated In-Context Learning: Iterative Refinement for Improved Answer Quality [62.72381208029899]
  In-context learning (ICL) enables language models to generate responses without modifying their parameters by leveraging examples provided in the input.<n>We propose Federated In-Context Learning (Fed-ICL), a general framework that enhances ICL through an iterative, collaborative process.<n>Fed-ICL progressively refines responses by leveraging multi-round interactions between clients and a central server, improving answer quality without the need to transmit model parameters.
  arXiv  Detail & Related papers  (2025-06-09T05:33:28Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application [54.984348122105516]
  Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework synergizes open-world knowledge with collaborative knowledge.<n>We propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge.
  arXiv  Detail & Related papers  (2024-05-07T04:00:30Z)
• A State-of-the-practice Release-readiness Checklist for Generative AI-based Software Products [8.986278918477595]
  This paper investigates the complexities of integrating Large Language Models into software products, with a focus on the challenges encountered for determining their readiness for release. Our systematic review of grey literature identifies common challenges in deploying LLMs, ranging from pre-training and fine-tuning to user experience considerations. The study introduces a comprehensive checklist designed to guide practitioners in evaluating key release readiness aspects such as performance, monitoring, and deployment strategies.
  arXiv  Detail & Related papers  (2024-03-27T19:02:56Z)
• Rethinking Machine Unlearning for Large Language Models [85.92660644100582]
  We explore machine unlearning in the domain of large language models (LLMs)<n>This initiative aims to eliminate undesirable data influence (e.g., sensitive or illegal information) and the associated model capabilities.
  arXiv  Detail & Related papers  (2024-02-13T20:51:58Z)
• Towards Generating Executable Metamorphic Relations Using Large Language Models [46.26208489175692]
  We propose an approach for automatically deriving executable MRs from requirements using large language models (LLMs) To assess the feasibility of our approach, we conducted a questionnaire-based survey in collaboration with Siemens Industry Software.
  arXiv  Detail & Related papers  (2024-01-30T13:52:47Z)
• Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey [7.164324501049983]
  Requirements Engineering (RE) can help address many problems when engineering Machine Learning-enabled systems. We conducted a survey to gather practitioner insights into the status quo and problems of RE in ML-enabled systems. We found significant differences in RE practices within ML projects.
  arXiv  Detail & Related papers  (2023-10-10T15:53:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.