AutonomyLens: A Self-Evolving Simulation-Based Testing Loop for Autonomous Systems

• URL: http://arxiv.org/abs/2604.11672v1
• Date: Mon, 13 Apr 2026 16:18:25 GMT
• Title: AutonomyLens: A Self-Evolving Simulation-Based Testing Loop for Autonomous Systems
• Abstract summary: AutonomyLens is a framework that integrates scenario specification, simulation execution, and telemetry analysis into a unified validation workflow.<n>We describe the early-stage design of AutonomyLens, discuss key challenges in building integrated validation for autonomous systems, and outline how such an approach can improve traceability, scalability, and autonomy in validation.
• Score: 8.335091665683924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Software engineering practices for validating autonomous cyber-physical systems (e.g., Uncrewed Aerial Vehicles) remain fragmented across scenario design, simulation execution, and telemetry analysis, limiting traceability between requirements, tests, and evidence. This fragmentation reduces reproducibility, slows debugging and iteration, and hinders systematic assurance under complex and evolving environmental conditions. We present AutonomyLens, an LLM-driven framework that integrates scenario specification, simulation execution, and telemetry analysis into a unified validation workflow. AutonomyLens enables developers to translate high-level validation intent into executable, temporally evolving scenarios, automatically run simulations, and perform context-aware analysis of resulting system behavior. The framework introduces (i) a structured representation for mission-level scenarios, (ii) an automated execution pipeline, (iii) analysis mechanisms that align telemetry with scenario context to produce actionable insights, and (iv) counterfactual scenario generation that closes the loop by refining and synthesizing new test cases from observed failures. We describe the early-stage design of AutonomyLens, discuss key challenges in building integrated validation workflows for autonomous systems, and outline how such an approach can improve traceability, reproducibility, and scalability in autonomy validation.

Related papers

• A Causal Probabilistic Framework for Perception-Informed Closed-Loop Simulation of Autonomous Driving [1.9798595819903433]
  Software-in-the-loop (SIL) simulation is a cornerstone for the validation of modern automotive safety functions.<n>This paper proposes a perception-informed SIL testing methodology that bridges the gap between ground-truth simulation and real-world perception behavior.
  arXiv  Detail & Related papers  (2026-06-05T11:53:43Z)
• AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration [49.050688065974605]
  Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
  arXiv  Detail & Related papers  (2026-04-01T15:17:45Z)
• Pitfalls in Evaluating Interpretability Agents [91.49742416116635]
  We build an agentic system in which a research agent iteratively designs experiments and refines hypotheses.<n>Our work demonstrates fundamental challenges in evaluating complex automated interpretability systems.
  arXiv  Detail & Related papers  (2026-03-20T16:27:17Z)
• A Unified Experimental Architecture for Informative Path Planning: from Simulation to Deployment with GuadalPlanner [69.43049144653882]
  This paper introduces a unified architecture that decouples high-level decision-making from vehicle-specific control.<n>The proposed architecture is realized through GuadalPlanner, which defines standardized interfaces between planning, sensing, and vehicle execution.
  arXiv  Detail & Related papers  (2026-02-11T10:02:31Z)
• Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generation [8.067859101380389]
  Non-executable or physically inconsistent outputs remain prevalent under stringent engineering constraints.<n>A framework for physics-consistent automatic building modeling is proposed.<n>CivilInstruct is introduced as a domain-specific dataset that formalizes structural engineering knowledge and constraint reasoning.<n> MBEval is presented as a verification-driven benchmark that evaluates executability and structural dynamics consistency.
  arXiv  Detail & Related papers  (2026-02-06T06:57:04Z)
• ARTIS: Agentic Risk-Aware Test-Time Scaling via Iterative Simulation [72.78362530982109]
  ARTIS, Agentic Risk-Aware Test-Time Scaling via Iterative Simulation, is a framework that decouples exploration from commitment.<n>We show that naive LLM-based simulators struggle to capture rare but high-impact failure modes.<n>We introduce a risk-aware tool simulator that emphasizes fidelity on failure-inducing actions.
  arXiv  Detail & Related papers  (2026-02-02T06:33:22Z)
• LLM-Agents Driven Automated Simulation Testing and Analysis of small Uncrewed Aerial Systems [11.183147511573717]
  Thorough simulation testing is crucial for validating the correct behavior of small Uncrewed Aerial Systems.<n>Various sUAS simulation tools exist to support developers, but the entire process of creating, executing, and analyzing simulation tests remains a largely manual and cumbersome task.<n>We propose AutoSimTest, a framework where multiple LLM agents collaborate to support the sUAS simulation testing process.
  arXiv  Detail & Related papers  (2025-01-21T03:42:21Z)
• GraphSCENE: On-Demand Critical Scenario Generation for Autonomous Vehicles in Simulation [11.896059467313668]
  This work introduces a novel method that generates dynamic temporal scene graphs corresponding to diverse traffic scenarios, on-demand, tailored to user-defined preferences.<n>A temporal Graph Neural Network (GNN) model learns to predict relationships between ego-vehicle agents and static structures, guided by real-world interaction patterns.<n>We render the predicted scenarios in simulation to further demonstrate their effectiveness as testing environments for AV agents.
  arXiv  Detail & Related papers  (2024-10-17T13:02:06Z)
• Coupled Requirements-driven Testing of CPS: From Simulation To Reality [5.7736484832934325]
  Failures in safety-critical Cyber-Physical Systems (CPS) can lead to severe incidents impacting physical infrastructure or even harming humans.<n>Current simulation and field testing practices, particularly in the domain of small Unmanned Aerial Systems (sUAS), are ad-hoc and lack a thorough, structured testing process.<n>We have developed an initial framework for validating CPS, specifically focusing on sUAS and robotic applications.
  arXiv  Detail & Related papers  (2024-03-24T20:32:12Z)
• Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review [62.997667081978825]
  This review aims at providing a comprehensive vision of the main state-of-the-art libraries and frameworks for machine learning and data analytics available today. The main simulation, emulation, deployment systems, and testbeds for experimental research on the Edge-to-Cloud Continuum available today are also surveyed.
  arXiv  Detail & Related papers  (2022-04-29T08:06:05Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.