Automated Test-Case Generation for REST APIs Using Model Inference Search Heuristic

• URL: http://arxiv.org/abs/2412.03420v2
• Date: Thu, 30 Jan 2025 12:35:06 GMT
• Title: Automated Test-Case Generation for REST APIs Using Model Inference Search Heuristic
• Authors: Clinton Cao, Annibale Panichella, Sicco Verwer,
• Abstract summary: EvoMaster is a tool that uses Evolutionary Algorithms (EAs) to automatically generate test cases for REST APIs. We propose a new search (MISH) that uses real-time automaton learning to guide the automated test case generation process. MISH learns a representation of the systemwide behavior, allowing to define the fitness of a test case based on the path it traverses within the inferred.
• Score: 15.625240669567479
• License:
• Abstract: The rising popularity of the microservice architectural style has led to a growing demand for automated testing approaches tailored to these systems. EvoMaster is a state-of-the-art tool that uses Evolutionary Algorithms (EAs) to automatically generate test cases for microservices' REST APIs. One limitation of these EAs is the use of unit-level search heuristics, such as branch distances, which focus on fine-grained code coverage and may not effectively capture the complex, interconnected behaviors characteristic of system-level testing. To address this limitation, we propose a new search heuristic (MISH) that uses real-time automaton learning to guide the test case generation process. We capture the sequential call patterns exhibited by a test case by learning an automaton from the stream of log events outputted by different microservices within the same system. Therefore, MISH learns a representation of the systemwide behavior, allowing us to define the fitness of a test case based on the path it traverses within the inferred automaton. We empirically evaluate MISH's effectiveness on six real-world benchmark microservice applications and compare it against a state-of-the-art technique, MOSA, for testing REST APIs. Our evaluation shows promising results for using MISH to guide the automated test case generation within EvoMaster.

Related papers

• LlamaRestTest: Effective REST API Testing with Small Language Models [50.058600784556816]
  We present LlamaRestTest, a novel approach that employs two custom LLMs to generate realistic test inputs. LlamaRestTest surpasses state-of-the-art tools in code coverage and error detection, even with RESTGPT-enhanced specifications.
  arXiv  Detail & Related papers  (2025-01-15T05:51:20Z)
• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChat [8.80569452545511]
  We introduce CAT to create cost-effective UI automation tests for industry apps by combining machine learning and Large Language Models. CAT then employs machine learning techniques, with LLMs serving as a complementary, to map the target element on the UI screen. Our evaluations on the WeChat testing dataset demonstrate the CAT's performance and cost-effectiveness, achieving 90% UI automation with $0.34 cost.
  arXiv  Detail & Related papers  (2024-09-12T08:25:33Z)
• KAT: Dependency-aware Automated API Testing with Large Language Models [1.7264233311359707]
  KAT (Katalon API Testing) is a novel AI-driven approach that autonomously generates test cases to validate APIs. Our evaluation of KAT using 12 real-world services shows that it can improve validation coverage, detect more undocumented status codes, and reduce false positives in these services.
  arXiv  Detail & Related papers  (2024-07-14T14:48:18Z)
• Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars [66.823588073584]
  Large language models (LLMs) have shown impressive capabilities in real-world applications. The quality of these exemplars in the prompt greatly impacts performance. Existing methods fail to adequately account for the impact of exemplar ordering on the performance.
  arXiv  Detail & Related papers  (2024-05-25T08:23:05Z)
• Automating REST API Postman Test Cases Using LLM [0.0]
  This research paper is dedicated to the exploration and implementation of an automated approach to generate test cases using Large Language Models. The methodology integrates the use of Open AI to enhance the efficiency and effectiveness of test case generation. The model that is developed during the research is trained using manually collected postman test cases or instances for various Rest APIs.
  arXiv  Detail & Related papers  (2024-04-16T15:53:41Z)
• Towards a Complete Metamorphic Testing Pipeline [56.75969180129005]
  Metamorphic Testing (MT) addresses the test oracle problem by examining the relationships between input-output pairs in consecutive executions of the System Under Test (SUT) These relations, known as Metamorphic Relations (MRs), specify the expected output changes resulting from specific input changes. Our research aims to develop methods and tools that assist testers in generating MRs, defining constraints, and providing explainability for MR outcomes.
  arXiv  Detail & Related papers  (2023-09-30T10:49:22Z)
• Adaptive REST API Testing with Reinforcement Learning [54.68542517176757]
  Current testing tools lack efficient exploration mechanisms, treating all operations and parameters equally. Current tools struggle when response schemas are absent in the specification or exhibit variants. We present an adaptive REST API testing technique incorporates reinforcement learning to prioritize operations during exploration.
  arXiv  Detail & Related papers  (2023-09-08T20:27:05Z)
• AmbieGen: A Search-based Framework for Autonomous Systems Testing [12.93632948681342]
  AmbieGen is a search-based test case generation framework for autonomous systems. It supports test case generation for autonomous robots and autonomous car lane keeping assist systems.
  arXiv  Detail & Related papers  (2023-01-01T23:42:32Z)
• Improving Test Case Generation for REST APIs Through Hierarchical Clustering [14.064310383770243]
  In the last decade, tools and approaches have been proposed to automate the creation of system-level test cases for APIs. One of the limiting factors of evolutionary algorithms (EAs) is that the genetic operators are fully randomized. This paper proposes a new approach that uses agglomerative hierarchical clustering (AHC) to infer a linkage tree model.
  arXiv  Detail & Related papers  (2021-09-14T12:57:49Z)
• Induction and Exploitation of Subgoal Automata for Reinforcement Learning [75.55324974788475]
  We present ISA, an approach for learning and exploiting subgoals in episodic reinforcement learning (RL) tasks. ISA interleaves reinforcement learning with the induction of a subgoal automaton, an automaton whose edges are labeled by the task's subgoals. A subgoal automaton also consists of two special states: a state indicating the successful completion of the task, and a state indicating that the task has finished without succeeding.
  arXiv  Detail & Related papers  (2020-09-08T16:42:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.