Dealing with Data for RE: Mitigating Challenges while using NLP and Generative AI

• URL: http://arxiv.org/abs/2402.16977v2
• Date: Wed, 28 Feb 2024 11:12:07 GMT
• Title: Dealing with Data for RE: Mitigating Challenges while using NLP and Generative AI
• Authors: Smita Ghaisas and Anmol Singhal
• Abstract summary: Book chapter explores the evolving landscape of Software Engineering in general, and Requirements Engineering (RE) in particular. We discuss challenges that arise while integrating Natural Language Processing (NLP) and generative AI into enterprise-critical software systems. Book provides practical insights, solutions, and examples to equip readers with the knowledge and tools necessary.
• Score: 2.9189409618561966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Across the dynamic business landscape today, enterprises face an ever-increasing range of challenges. These include the constantly evolving regulatory environment, the growing demand for personalization within software applications, and the heightened emphasis on governance. In response to these multifaceted demands, large enterprises have been adopting automation that spans from the optimization of core business processes to the enhancement of customer experiences. Indeed, Artificial Intelligence (AI) has emerged as a pivotal element of modern software systems. In this context, data plays an indispensable role. AI-centric software systems based on supervised learning and operating at an industrial scale require large volumes of training data to perform effectively. Moreover, the incorporation of generative AI has led to a growing demand for adequate evaluation benchmarks. Our experience in this field has revealed that the requirement for large datasets for training and evaluation introduces a host of intricate challenges. This book chapter explores the evolving landscape of Software Engineering (SE) in general, and Requirements Engineering (RE) in particular, in this era marked by AI integration. We discuss challenges that arise while integrating Natural Language Processing (NLP) and generative AI into enterprise-critical software systems. The chapter provides practical insights, solutions, and examples to equip readers with the knowledge and tools necessary for effectively building solutions with NLP at their cores. We also reflect on how these text data-centric tasks sit together with the traditional RE process. We also highlight new RE tasks that may be necessary for handling the increasingly important text data-centricity involved in developing software systems.

Related papers

• Prioritizing Software Requirements Using Large Language Models [3.9422957660677476]
  This article focuses on requirements engineering, typically seen as the initial phase of software development. The challenge of identifying requirements and satisfying all stakeholders within time and budget constraints remains significant. This study introduces a web-based software tool utilizing AI agents and prompt engineering to automate task prioritization.
  arXiv  Detail & Related papers  (2024-04-05T15:20:56Z)
• Prompted Software Engineering in the Era of AI Models [1.450405446885067]
  This paper introduces prompted software engineering (PSE), which integrates prompt engineering to build effective prompts for language-based AI models. PSE enables the use of AI models in software development to produce high-quality software with fewer resources, automating tedious tasks and allowing developers to focus on more innovative aspects.
  arXiv  Detail & Related papers  (2023-09-07T20:40:04Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• Automotive Perception Software Development: An Empirical Investigation into Data, Annotation, and Ecosystem Challenges [10.649193588119985]
  Software that contains machine learning algorithms is an integral part of automotive perception. The development of such software, specifically the training and validation of the machine learning components, require large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive software components.
  arXiv  Detail & Related papers  (2023-03-10T14:29:06Z)
• Privacy Adhering Machine Un-learning in NLP [66.17039929803933]
  In real world industry use Machine Learning to build models on user data. Such mandates require effort both in terms of data as well as model retraining. continuous removal of data and model retraining steps do not scale. We propose textitMachine Unlearning to tackle this challenge.
  arXiv  Detail & Related papers  (2022-12-19T16:06:45Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Federated Learning for Industrial Internet of Things in Future Industries [106.13524161081355]
  The Industrial Internet of Things (IIoT) offers promising opportunities to transform the operation of industrial systems. Recently, artificial intelligence (AI) has been widely utilized for realizing intelligent IIoT applications. Federated Learning (FL) is particularly attractive for intelligent IIoT networks by coordinating multiple IIoT devices and machines to perform AI training at the network edge.
  arXiv  Detail & Related papers  (2021-05-31T01:02:59Z)
• Engineering an Intelligent Essay Scoring and Feedback System: An Experience Report [1.5168188294440734]
  We describe an exploratory system for assessing the quality of essays supplied by customers of a specialized recruitment support service. The problem domain is challenging because the open-ended customer-supplied source text has considerable scope for ambiguity and error. There is also a need to incorporate specialized business domain knowledge into the intelligent processing systems.
  arXiv  Detail & Related papers  (2021-03-25T03:46:05Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)
• Software engineering for artificial intelligence and machine learning software: A systematic literature review [6.681725960709127]
  This study aims to investigate how software engineering has been applied in the development of AI/ML systems. Main challenges faced by professionals are in areas of testing, AI software quality, and data management.
  arXiv  Detail & Related papers  (2020-11-07T11:06:28Z)
• Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology [53.063411515511056]
  We propose a process model for the development of machine learning applications. The first phase combines business and data understanding as data availability oftentimes affects the feasibility of the project. The sixth phase covers state-of-the-art approaches for monitoring and maintenance of a machine learning applications.
  arXiv  Detail & Related papers  (2020-03-11T08:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.