JTA: Joint Testability Architecture for Scenario-Based Validation of Safety-Critical Software

• URL: http://arxiv.org/abs/2608.05594v1
• Date: Thu, 06 Aug 2026 04:35:38 GMT
• Title: JTA: Joint Testability Architecture for Scenario-Based Validation of Safety-Critical Software
• Abstract summary: Joint Testability Architecture (JTA) characterizes validation capability along three dimensions--controllability, observability, and isolability--and organizes them through three domains.<n> JTA provides an architectural basis for modeling, designing, and assessing scenario-based validation in safety-critical software.
• Score: 3.90115985145439
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Validation adequacy in safety-critical software depends on more than the system under test. Critical scenarios must be constructed under controlled conditions, execution evidence must be aligned into verdict-ready form, and abnormal outcomes must be attributable to actionable causes. Existing testability research remains largely artifact-centric and offers little architectural support for reasoning about the combined capability of the scenario, the test system, and the system under test. Joint Testability Architecture (JTA) addresses this gap by treating those three elements as a single object of analysis and design. It characterizes validation capability along three dimensions--controllability, observability, and isolability--and organizes them through three domains, three bridges, and an analysis-design-evaluation-refinement loop. JTA also introduces scenario contracts, joint capability assessment, validation blind-spot identification, and bridge-oriented design actions that map capability gaps to concrete improvements in control points, evidence organization, and attribution boundaries. An illustrative analysis of ArduPilot failsafe validation shows that link-loss scenarios are comparatively mature, whereas state-estimation anomaly scenarios remain harder to validate because evidence alignment and attribution semantics are weaker. JTA is not a replacement for existing testing or safety-analysis techniques; it provides an architectural basis for modeling, designing, and assessing scenario-based validation in safety-critical software.

Related papers

• FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models [45.90905049052029]
  We propose FLY-EVAL++, an evidence-driven evaluation protocol for large language models in safety-critical environments.<n>We instantiate FLY-EVAL++ for Flight Trajectory and Attitude Prediction (FTAP) by extending the PilotBench setting with history-conditioned and multi-step prediction tasks.<n>Our results show that evaluation in safety-critical domains should measure constraint satisfaction and structured validity explicitly rather than rely on accuracy-centric reporting alone.
  arXiv  Detail & Related papers  (2026-09-03T16:00:02Z)
• Towards Trustworthy Embodied Intelligence: A Systems Framework and Graded Trustworthiness Levels [53.63220028820581]
  Embodied intelligence integrates learned perception and decision making with real-time computation, control, and physical interaction.<n>We define trustworthy embodied intelligence as the sustained capacity to execute specified tasks reliably.
  arXiv  Detail & Related papers  (2026-07-28T17:50:44Z)
• Gatekeepers and Hallucinations: A Layered Evaluation Framework for LLM-Driven Quantum Circuit Generation [0.0]
  We present a layered evaluation framework for materials-informed Variational Quantum Eigensolver (VQE) circuit generation.<n>We surface a taxonomy of five distinct LLM failure modes (geometry hallucination, nonexistent API usage, integration failures, constraint violations, and plausible-but-unverifiable output)<n>A forensic audit of the evaluation platform's own source code further establishes that two apparent model failures originated in the harness through silent fallback-template substitution.
  arXiv  Detail & Related papers  (2026-06-16T19:17:54Z)
• What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs [52.50210189669399]
  Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs)<n>We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation.<n>We show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods.
  arXiv  Detail & Related papers  (2026-06-01T03:18:01Z)
• Referential Security as a New Paradigm for AI Evaluations [5.748110232428846]
  We propose referential security as a new paradigm for AI evaluation.<n>This approach reframes model identity as an empirically verifiable property and separates referential stability from the substantive security claims it conditions.
  arXiv  Detail & Related papers  (2026-05-25T10:24:10Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations [46.98103424349245]
  Existing safety evaluation benchmarks and frameworks are often limited by an imbalanced integration of core components.<n>We present TeleAI-Safety, a modular and reproducible framework coupled with a systematic benchmark for rigorous safety evaluation.<n>With a curated attack corpus of 342 samples spanning 12 distinct risk categories, the TeleAI-Safety benchmark conducts extensive evaluations across 14 target models.
  arXiv  Detail & Related papers  (2025-12-05T07:23:30Z)
• A Requirements-Driven Platform for Validating Field Operations of Small Uncrewed Aerial Vehicles [48.67061953896227]
  DroneReqValidator (DRV) allows sUAS developers to define the operating context, configure multi-sUAS mission requirements, specify safety properties, and deploy their own custom sUAS applications in a high-fidelity 3D environment. The DRV Monitoring system collects runtime data from sUAS and the environment, analyzes compliance with safety properties, and captures violations.
  arXiv  Detail & Related papers  (2023-07-01T02:03:49Z)
• A Verification Framework for Component-Based Modeling and Simulation Putting the pieces together [0.0]
  The proposed verification framework provides methods, techniques and tool support for verifying composability at its different levels. In particular we focus on the Dynamic-Semantic Composability level due to its significance in the overall composability correctness and also due to the level of difficulty it poses in the process.
  arXiv  Detail & Related papers  (2023-01-08T18:53:28Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• A Review of Testing Object-Based Environment Perception for Safe Automated Driving [0.0]
  Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance.
  arXiv  Detail & Related papers  (2021-02-16T21:40:39Z)
• Efficient statistical validation with edge cases to evaluate Highly Automated Vehicles [6.198523595657983]
  The widescale deployment of Autonomous Vehicles seems to be imminent despite many safety challenges that are yet to be resolved. Existing standards focus on deterministic processes where the validation requires only a set of test cases that cover the requirements. This paper presents a new approach to compute the statistical characteristics of a system's behaviour by biasing automatically generated test cases towards the worst case scenarios.
  arXiv  Detail & Related papers  (2020-03-04T04:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.