Carving UI Tests to Generate API Tests and API Specification
- URL: http://arxiv.org/abs/2305.14692v1
- Date: Wed, 24 May 2023 03:53:34 GMT
- Title: Carving UI Tests to Generate API Tests and API Specification
- Authors: Rahulkrishna Yandrapally, Saurabh Sinha, Rachel Tzoref-Brill, Ali
Mesbah
- Abstract summary: API-level testing can play an important role, in-between unit-level testing and UI-level (or end-to-end) testing.
Existing API testing tools require API specifications, which often may not be available or, when available, be inconsistent with the API implementation.
We present an approach that leverages UI testing to enable API-level testing for web applications.
- Score: 8.743426215048451
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Modern web applications make extensive use of API calls to update the UI
state in response to user events or server-side changes. For such applications,
API-level testing can play an important role, in-between unit-level testing and
UI-level (or end-to-end) testing. Existing API testing tools require API
specifications (e.g., OpenAPI), which often may not be available or, when
available, be inconsistent with the API implementation, thus limiting the
applicability of automated API testing to web applications. In this paper, we
present an approach that leverages UI testing to enable API-level testing for
web applications. Our technique navigates the web application under test and
automatically generates an API-level test suite, along with an OpenAPI
specification that describes the application's server-side APIs (for REST-based
web applications). A key element of our solution is a dynamic approach for
inferring API endpoints with path parameters via UI navigation and directed API
probing. We evaluated the technique for its accuracy in inferring API
specifications and the effectiveness of the "carved" API tests. Our results on
seven open-source web applications show that the technique achieves 98%
precision and 56% recall in inferring endpoints. The carved API tests, when
added to test suites generated by two automated REST API testing tools,
increase statement coverage by 52% and 29% and branch coverage by 99% and 75%,
on average. The main benefits of our technique are: (1) it enables API-level
testing of web applications in cases where existing API testing tools are
inapplicable and (2) it creates API-level test suites that cover server-side
code efficiently while exercising APIs as they would be invoked from an
application's web UI, and that can augment existing API test suites.
Related papers
- KAT: Dependency-aware Automated API Testing with Large Language Models [1.7264233311359707]
KAT (Katalon API Testing) is a novel AI-driven approach that autonomously generates test cases to validate APIs.
Our evaluation of KAT using 12 real-world services shows that it can improve validation coverage, detect more undocumented status codes, and reduce false positives in these services.
arXiv Detail & Related papers (2024-07-14T14:48:18Z) - WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment [49.00213183302225]
We propose a framework to induce new APIs by grounding wikiHow instruction to situated agent policies.
Inspired by recent successes in large language models (LLMs) for embodied planning, we propose a few-shot prompting to steer GPT-4.
arXiv Detail & Related papers (2024-07-10T15:52:44Z) - A Solution-based LLM API-using Methodology for Academic Information Seeking [49.096714812902576]
SoAy is a solution-based LLM API-using methodology for academic information seeking.
It uses code with a solution as the reasoning method, where a solution is a pre-constructed API calling sequence.
Results show a 34.58-75.99% performance improvement compared to state-of-the-art LLM API-based baselines.
arXiv Detail & Related papers (2024-05-24T02:44:14Z) - You Can REST Now: Automated Specification Inference and Black-Box
Testing of RESTful APIs with Large Language Models [8.753312212588371]
manually documenting APIs is a time-consuming and error-prone task, resulting in unavailable, incomplete, or imprecise documentation.
Recently, Large Language Models (LLMs) have demonstrated exceptional abilities to automate tasks based on their colossal training data.
We present RESTSpecIT, the first automated API specification inference and black-box testing approach.
arXiv Detail & Related papers (2024-02-07T18:55:41Z) - Leveraging Large Language Models to Improve REST API Testing [51.284096009803406]
RESTGPT takes as input an API specification, extracts machine-interpretable rules, and generates example parameter values from natural-language descriptions in the specification.
Our evaluations indicate that RESTGPT outperforms existing techniques in both rule extraction and value generation.
arXiv Detail & Related papers (2023-12-01T19:53:23Z) - Exploring Behaviours of RESTful APIs in an Industrial Setting [0.43012765978447565]
We propose a set of behavioural properties, common to REST APIs, which are used to generate examples of behaviours that these APIs exhibit.
These examples can be used both (i) to further the understanding of the API and (ii) as a source of automatic test cases.
Our approach can generate examples deemed relevant for understanding the system and for a source of test generation by practitioners.
arXiv Detail & Related papers (2023-10-26T11:33:11Z) - APICom: Automatic API Completion via Prompt Learning and Adversarial
Training-based Data Augmentation [6.029137544885093]
API recommendation is the process of assisting developers in finding the required API among numerous candidate APIs.
Previous studies mainly modeled API recommendation as the recommendation task, and developers may not yet be able to find what they need.
Motivated by the neural machine translation research domain, we can model this problem as the generation task.
We propose a novel approach APICom based on prompt learning, which can generate API related to the query according to the prompts.
arXiv Detail & Related papers (2023-09-13T15:31:50Z) - Adaptive REST API Testing with Reinforcement Learning [54.68542517176757]
Current testing tools lack efficient exploration mechanisms, treating all operations and parameters equally.
Current tools struggle when response schemas are absent in the specification or exhibit variants.
We present an adaptive REST API testing technique incorporates reinforcement learning to prioritize operations during exploration.
arXiv Detail & Related papers (2023-09-08T20:27:05Z) - Automatic Unit Test Generation for Deep Learning Frameworks based on API
Knowledge [11.523398693942413]
We propose MUTester to generate unit test cases for APIs of deep learning frameworks.
We first propose a set of 18 rules for mining API constraints from the API documents.
We then use the frequent itemset mining technique to mine the API usage patterns from a large corpus of machine learning API related code fragments.
arXiv Detail & Related papers (2023-07-01T18:34:56Z) - Evaluating Embedding APIs for Information Retrieval [51.24236853841468]
We evaluate the capabilities of existing semantic embedding APIs on domain generalization and multilingual retrieval.
We find that re-ranking BM25 results using the APIs is a budget-friendly approach and is most effective in English.
For non-English retrieval, re-ranking still improves the results, but a hybrid model with BM25 works best, albeit at a higher cost.
arXiv Detail & Related papers (2023-05-10T16:40:52Z) - Simple Transparent Adversarial Examples [65.65977217108659]
We introduce secret embedding and transparent adversarial examples as a simpler way to evaluate robustness.
As a result, they pose a serious threat where APIs are used for high-stakes applications.
arXiv Detail & Related papers (2021-05-20T11:54:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.