Carving UI Tests to Generate API Tests and API Specification

• URL: http://arxiv.org/abs/2305.14692v1
• Date: Wed, 24 May 2023 03:53:34 GMT
• Title: Carving UI Tests to Generate API Tests and API Specification
• Authors: Rahulkrishna Yandrapally, Saurabh Sinha, Rachel Tzoref-Brill, Ali Mesbah
• Abstract summary: API-level testing can play an important role, in-between unit-level testing and UI-level (or end-to-end) testing. Existing API testing tools require API specifications, which often may not be available or, when available, be inconsistent with the API implementation. We present an approach that leverages UI testing to enable API-level testing for web applications.
• Score: 8.743426215048451
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern web applications make extensive use of API calls to update the UI state in response to user events or server-side changes. For such applications, API-level testing can play an important role, in-between unit-level testing and UI-level (or end-to-end) testing. Existing API testing tools require API specifications (e.g., OpenAPI), which often may not be available or, when available, be inconsistent with the API implementation, thus limiting the applicability of automated API testing to web applications. In this paper, we present an approach that leverages UI testing to enable API-level testing for web applications. Our technique navigates the web application under test and automatically generates an API-level test suite, along with an OpenAPI specification that describes the application's server-side APIs (for REST-based web applications). A key element of our solution is a dynamic approach for inferring API endpoints with path parameters via UI navigation and directed API probing. We evaluated the technique for its accuracy in inferring API specifications and the effectiveness of the "carved" API tests. Our results on seven open-source web applications show that the technique achieves 98% precision and 56% recall in inferring endpoints. The carved API tests, when added to test suites generated by two automated REST API testing tools, increase statement coverage by 52% and 29% and branch coverage by 99% and 75%, on average. The main benefits of our technique are: (1) it enables API-level testing of web applications in cases where existing API testing tools are inapplicable and (2) it creates API-level test suites that cover server-side code efficiently while exercising APIs as they would be invoked from an application's web UI, and that can augment existing API test suites.

Related papers

• Utilizing API Response for Test Refinement [2.8002188463519944]
  This paper proposes a dynamic test refinement approach that leverages the response message. Using an intelligent agent, the approach adds constraints to the API specification that are further used to generate a test scenario. The proposed approach led to a decrease in the number of 4xx responses, taking a step closer to generating more realistic test cases.
  arXiv  Detail & Related papers  (2025-01-30T05:26:32Z)
• LlamaRestTest: Effective REST API Testing with Small Language Models [50.058600784556816]
  We present LlamaRestTest, a novel approach that employs two custom LLMs to generate realistic test inputs. LlamaRestTest surpasses state-of-the-art tools in code coverage and error detection, even with RESTGPT-enhanced specifications.
  arXiv  Detail & Related papers  (2025-01-15T05:51:20Z)
• ExploraCoder: Advancing code generation for multiple unseen APIs via planning and chained exploration [70.26807758443675]
  ExploraCoder is a training-free framework that empowers large language models to invoke unseen APIs in code solution. We show that ExploraCoder significantly improves performance for models lacking prior API knowledge, achieving an absolute increase of 11.24% over niave RAG approaches and 14.07% over pretraining methods in pass@10.
  arXiv  Detail & Related papers  (2024-12-06T19:00:15Z)
• A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs [46.65963514391019]
  We present AutoRestTest, the first black-box tool to adopt a dependency-embedded multi-agent approach for REST API testing. Our approach treats REST API testing as a separable problem, where four agents collaborate to optimize API exploration. Our evaluation of AutoRestTest on 12 real-world REST services shows that it outperforms the four leading black-box REST API testing tools.
  arXiv  Detail & Related papers  (2024-11-11T16:20:27Z)
• Beyond Browsing: API-Based Web Agents [58.39129004543844]
  API-based agents outperform web browsing agents in experiments on WebArena. Hybrid Agents out-perform both others nearly uniformly across tasks. Results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.
  arXiv  Detail & Related papers  (2024-10-21T19:46:06Z)
• A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How [53.65636914757381]
  API suggestion is a critical task in modern software development. Recent advancements in large code models (LCMs) have shown promise in the API suggestion task.
  arXiv  Detail & Related papers  (2024-09-20T03:12:35Z)
• DeepREST: Automated Test Case Generation for REST APIs Exploiting Deep Reinforcement Learning [5.756036843502232]
  This paper introduces DeepREST, a novel black-box approach for automatically testing REST APIs. It leverages deep reinforcement learning to uncover implicit API constraints, that is, constraints hidden from API documentation. Our empirical validation suggests that the proposed approach is very effective in achieving high test coverage and fault detection.
  arXiv  Detail & Related papers  (2024-08-16T08:03:55Z)
• WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment [49.00213183302225]
  We propose a framework to induce new APIs by grounding wikiHow instruction to situated agent policies. Inspired by recent successes in large language models (LLMs) for embodied planning, we propose a few-shot prompting to steer GPT-4.
  arXiv  Detail & Related papers  (2024-07-10T15:52:44Z)
• You Can REST Now: Automated Specification Inference and Black-Box Testing of RESTful APIs with Large Language Models [8.753312212588371]
  manually documenting APIs is a time-consuming and error-prone task, resulting in unavailable, incomplete, or imprecise documentation. Recently, Large Language Models (LLMs) have demonstrated exceptional abilities to automate tasks based on their colossal training data. We present RESTSpecIT, the first automated API specification inference and black-box testing approach.
  arXiv  Detail & Related papers  (2024-02-07T18:55:41Z)
• Exploring Behaviours of RESTful APIs in an Industrial Setting [0.43012765978447565]
  We propose a set of behavioural properties, common to REST APIs, which are used to generate examples of behaviours that these APIs exhibit. These examples can be used both (i) to further the understanding of the API and (ii) as a source of automatic test cases. Our approach can generate examples deemed relevant for understanding the system and for a source of test generation by practitioners.
  arXiv  Detail & Related papers  (2023-10-26T11:33:11Z)
• Automatic Unit Test Generation for Deep Learning Frameworks based on API Knowledge [11.523398693942413]
  We propose MUTester to generate unit test cases for APIs of deep learning frameworks. We first propose a set of 18 rules for mining API constraints from the API documents. We then use the frequent itemset mining technique to mine the API usage patterns from a large corpus of machine learning API related code fragments.
  arXiv  Detail & Related papers  (2023-07-01T18:34:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.