Reasoning-Based Software Testing
- URL: http://arxiv.org/abs/2303.01302v1
- Date: Thu, 2 Mar 2023 14:27:21 GMT
- Title: Reasoning-Based Software Testing
- Authors: Luca Giamattei, Roberto Pietrantuono, Stefano Russo
- Abstract summary: Reasoning-Based Software Testing (RBST) is a new way of thinking at the testing problem as a causal reasoning task.
We claim that causal reasoning more naturally emulates the process that a human would do to ''smartly" search the space.
Preliminary results reported in this paper are promising.
- Score: 9.341830361844337
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: With software systems becoming increasingly pervasive and autonomous, our
ability to test for their quality is severely challenged. Many systems are
called to operate in uncertain and highly-changing environment, not rarely
required to make intelligent decisions by themselves. This easily results in an
intractable state space to explore at testing time. The state-of-the-art
techniques try to keep the pace, e.g., by augmenting the tester's intuition
with some form of (explicit or implicit) learning from observations to search
this space efficiently. For instance, they exploit historical data to drive the
search (e.g., ML-driven testing) or the tests execution data itself (e.g.,
adaptive or search-based testing). Despite the indubitable advances, the need
for smartening the search in such a huge space keeps to be pressing.
We introduce Reasoning-Based Software Testing (RBST), a new way of thinking
at the testing problem as a causal reasoning task. Compared to mere
intuition-based or state-of-the-art learning-based strategies, we claim that
causal reasoning more naturally emulates the process that a human would do to
''smartly" search the space. RBST aims to mimic and amplify, with the power of
computation, this ability. The conceptual leap can pave the ground to a new
trend of techniques, which can be variously instantiated from the proposed
framework, by exploiting the numerous tools for causal discovery and inference.
Preliminary results reported in this paper are promising.
Related papers
- Geospatial Trajectory Generation via Efficient Abduction: Deployment for Independent Testing [1.8877926393541125]
We show that we can abduce movement trajectories efficiently through an informed (i.e., A*) search.
We also report on our own experiments showing that we not only provide exact results but also scale to very large scenarios.
arXiv Detail & Related papers (2024-07-08T23:11:47Z) - "Turing Tests" For An AI Scientist [0.0]
This paper proposes a "Turing test for an AI scientist" to assess whether an AI agent can conduct scientific research independently.
We propose seven benchmark tests that evaluate an AI agent's ability to make groundbreaking discoveries in various scientific domains.
arXiv Detail & Related papers (2024-05-22T05:14:27Z) - The Causal Chambers: Real Physical Systems as a Testbed for AI Methodology [10.81691411087626]
In some fields of AI, machine learning and statistics, the validation of new methods and algorithms is often hindered by the scarcity of suitable real-world datasets.
We have constructed two devices that allow us to quickly and inexpensively produce large datasets from non-trivial but well-understood physical systems.
arXiv Detail & Related papers (2024-04-17T13:00:52Z) - A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies.
Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance.
Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
arXiv Detail & Related papers (2023-10-11T15:21:40Z) - Brain-Inspired Computational Intelligence via Predictive Coding [89.6335791546526]
Predictive coding (PC) has shown promising performance in machine intelligence tasks.
PC can model information processing in different brain areas, can be used in cognitive control and robotics.
arXiv Detail & Related papers (2023-08-15T16:37:16Z) - Does AI for science need another ImageNet Or totally different
benchmarks? A case study of machine learning force fields [5.622820801789953]
AI for science (AI4S) aims to enhance the accuracy and speed of scientific computing tasks using machine learning methods.
Traditional AI benchmarking methods struggle to adapt to the unique challenges posed by AI4S because they assume data in training, testing, and future real-world queries are independent and identically distributed.
This paper investigates the need for a novel approach to effectively benchmark AI for science, using the machine learning force field (MLFF) as a case study.
arXiv Detail & Related papers (2023-08-11T08:06:58Z) - Constrained Adversarial Learning and its applicability to Automated
Software Testing: a systematic review [0.0]
This systematic review is focused on the current state-of-the-art of constrained data generation methods applied for adversarial learning and software testing.
It aims to guide researchers and developers to enhance testing tools with adversarial learning methods and improve the resilience and robustness of their digital systems.
arXiv Detail & Related papers (2023-03-14T00:27:33Z) - Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world.
Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts.
This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
arXiv Detail & Related papers (2021-12-17T16:28:06Z) - IQ-Learn: Inverse soft-Q Learning for Imitation [95.06031307730245]
imitation learning from a small amount of expert data can be challenging in high-dimensional environments with complex dynamics.
Behavioral cloning is a simple method that is widely used due to its simplicity of implementation and stable convergence.
We introduce a method for dynamics-aware IL which avoids adversarial training by learning a single Q-function.
arXiv Detail & Related papers (2021-06-23T03:43:10Z) - A Review of Uncertainty Quantification in Deep Learning: Techniques,
Applications and Challenges [76.20963684020145]
Uncertainty quantification (UQ) plays a pivotal role in reduction of uncertainties during both optimization and decision making processes.
Bizarre approximation and ensemble learning techniques are two most widely-used UQ methods in the literature.
This study reviews recent advances in UQ methods used in deep learning and investigates the application of these methods in reinforcement learning.
arXiv Detail & Related papers (2020-11-12T06:41:05Z) - AutoOD: Automated Outlier Detection via Curiosity-guided Search and
Self-imitation Learning [72.99415402575886]
Outlier detection is an important data mining task with numerous practical applications.
We propose AutoOD, an automated outlier detection framework, which aims to search for an optimal neural network model.
Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoOD achieves the best performance.
arXiv Detail & Related papers (2020-06-19T18:57:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.