Exploring API Behaviours Through Generated Examples
- URL: http://arxiv.org/abs/2308.15210v1
- Date: Tue, 29 Aug 2023 11:05:52 GMT
- Title: Exploring API Behaviours Through Generated Examples
- Authors: Stefan Karlsson, John Hughes, Robbert Jongeling, Adnan Causevic,
Daniel Sundmark
- Abstract summary: We present an approach to automatically generate relevant examples of behaviours of an API.
Our method can produce small and relevant examples that can help engineers to understand the system under exploration.
- Score: 0.768721532845575
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Understanding the behaviour of a system's API can be hard. Giving users
access to relevant examples of how an API behaves has been shown to make this
easier for them. In addition, such examples can be used to verify expected
behaviour or identify unwanted behaviours.
Methods for automatically generating examples have existed for a long time.
However, state-of-the-art methods rely on either white-box information, such as
source code, or on formal specifications of the system behaviour. But what if
you do not have access to either? e.g., when interacting with a third-party
API.
In this paper, we present an approach to automatically generate relevant
examples of behaviours of an API, without requiring either source code or a
formal specification of behaviour.
Evaluation on an industry-grade REST API shows that our method can produce
small and relevant examples that can help engineers to understand the system
under exploration.
Related papers
- A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How [53.65636914757381]
API suggestion is a critical task in modern software development.
Recent advancements in large code models (LCMs) have shown promise in the API suggestion task.
arXiv Detail & Related papers (2024-09-20T03:12:35Z) - DeepREST: Automated Test Case Generation for REST APIs Exploiting Deep Reinforcement Learning [5.756036843502232]
This paper introduces DeepREST, a novel black-box approach for automatically testing REST APIs.
It leverages deep reinforcement learning to uncover implicit API constraints, that is, constraints hidden from API documentation.
Our empirical validation suggests that the proposed approach is very effective in achieving high test coverage and fault detection.
arXiv Detail & Related papers (2024-08-16T08:03:55Z) - FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking [57.53742155914176]
API call generation is the cornerstone of large language models' tool-using ability.
Existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user's request.
We propose an output-side optimization approach called FANTASE to address these limitations.
arXiv Detail & Related papers (2024-07-18T23:44:02Z) - WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment [49.00213183302225]
We propose a framework to induce new APIs by grounding wikiHow instruction to situated agent policies.
Inspired by recent successes in large language models (LLMs) for embodied planning, we propose a few-shot prompting to steer GPT-4.
arXiv Detail & Related papers (2024-07-10T15:52:44Z) - Exploring the Impact of Source Code Linearity on the Programmers Comprehension of API Code Examples [0.0]
We investigated whether the (a) linearity and (b) length of the source code in API code examples affect users performance in terms of correctness and time spent.
We conducted an online controlled code comprehension experiment with 61 Java developers.
arXiv Detail & Related papers (2024-04-03T00:40:38Z) - SpeCrawler: Generating OpenAPI Specifications from API Documentation
Using Large Language Models [8.372941103284774]
SpeCrawler is a comprehensive system that generates OpenAPI Specifications from diverse API documentation.
The paper explores SpeCrawler's methodology, supported by empirical evidence and case studies.
arXiv Detail & Related papers (2024-02-18T15:33:24Z) - Exploring Behaviours of RESTful APIs in an Industrial Setting [0.43012765978447565]
We propose a set of behavioural properties, common to REST APIs, which are used to generate examples of behaviours that these APIs exhibit.
These examples can be used both (i) to further the understanding of the API and (ii) as a source of automatic test cases.
Our approach can generate examples deemed relevant for understanding the system and for a source of test generation by practitioners.
arXiv Detail & Related papers (2023-10-26T11:33:11Z) - Binding Language Models in Symbolic Languages [146.3027328556881]
Binder is a training-free neural-symbolic framework that maps the task input to a program.
In the parsing stage, Codex is able to identify the part of the task input that cannot be answerable by the original programming language.
In the execution stage, Codex can perform versatile functionalities given proper prompts in the API calls.
arXiv Detail & Related papers (2022-10-06T12:55:17Z) - Visual Transformer for Task-aware Active Learning [49.903358393660724]
We present a novel pipeline for pool-based Active Learning.
Our method exploits accessible unlabelled examples during training to estimate their co-relation with the labelled examples.
Visual Transformer models non-local visual concept dependency between labelled and unlabelled examples.
arXiv Detail & Related papers (2021-06-07T17:13:59Z) - An Empirical Comparison of Instance Attribution Methods for NLP [62.63504976810927]
We evaluate the degree to which different potential instance attribution agree with respect to the importance of training samples.
We find that simple retrieval methods yield training instances that differ from those identified via gradient-based methods.
arXiv Detail & Related papers (2021-04-09T01:03:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.