Querying Labeled Time Series Data with Scenario Programs

• URL: http://arxiv.org/abs/2511.10627v1
• Date: Fri, 14 Nov 2025 02:00:34 GMT
• Title: Querying Labeled Time Series Data with Scenario Programs
• Authors: Edward Kim, Devan Shanker, Varun Bharadwaj, Hongbeen Park, Jinkyu Kim, Hazem Torfah, Daniel J Fremont, Sanjit A Seshia,
• Abstract summary: Simulation-based testing has become a crucial complement to road testing for ensuring the safety of cyber physical systems.<n>Research efforts have been directed toward identifying failure scenarios within simulation environments.<n>Are the AV failure scenarios discovered in reproducible simulation on actual systems in the real world?
• Score: 16.919240746807812
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simulation-based testing has become a crucial complement to road testing for ensuring the safety of cyber physical systems (CPS). As a result, significant research efforts have been directed toward identifying failure scenarios within simulation environments. However, a critical question remains. Are the AV failure scenarios discovered in simulation reproducible on actual systems in the real world? The sim-to-real gap caused by differences between simulated and real sensor data means that failure scenarios identified in simulation might either be artifacts of synthetic sensor data or actual issues that also occur with real sensor data. To address this, an effective approach to validating simulated failure scenarios is to locate occurrences of these scenarios within real-world datasets and verify whether the failure persists on the datasets. To this end, we introduce a formal definition of how labeled time series sensor data can match an abstract scenario, represented as a scenario program using the Scenic probabilistic programming language. We present a querying algorithm that, given a scenario program and a labeled dataset, identifies the subset of data that matches the specified scenario. Our experiment shows that our algorithm is more accurate and orders of magnitude faster in querying scenarios than the state-of-the-art commercial vision large language models, and can scale with the duration of queried time series data.

Related papers

• Temporal Causal-based Simulation for Realistic Time-series Generation [1.49201581313345]
  Causal Discovery plays a pivotal role in revealing relationships among observed variables, particularly in the temporal setup.<n>Generation techniques depending on simplified assumptions on causal structure, effects and time, limit the quality and diversity of the simulated data.<n>We introduce Temporal Causal-based Simulation (TCS), a robust framework for generating realistic time-series data and their associated temporal causal graphs.
  arXiv  Detail & Related papers  (2025-06-02T10:59:48Z)
• GraphSCENE: On-Demand Critical Scenario Generation for Autonomous Vehicles in Simulation [11.896059467313668]
  This work introduces a novel method that generates dynamic temporal scene graphs corresponding to diverse traffic scenarios, on-demand, tailored to user-defined preferences.<n>A temporal Graph Neural Network (GNN) model learns to predict relationships between ego-vehicle agents and static structures, guided by real-world interaction patterns.<n>We render the predicted scenarios in simulation to further demonstrate their effectiveness as testing environments for AV agents.
  arXiv  Detail & Related papers  (2024-10-17T13:02:06Z)
• Querying Labeled Time Series Data with Scenario Programs [0.0]
  We propose a formal definition of what constitutes a match between a real-world labeled time series data item and a simulated scenario. We present a definition and algorithm for matching scalable beyond the autonomous vehicles domain.
  arXiv  Detail & Related papers  (2024-06-25T15:15:27Z)
• NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking [65.24988062003096]
  We present NAVSIM, a framework for benchmarking vision-based driving policies. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights.
  arXiv  Detail & Related papers  (2024-06-21T17:59:02Z)
• Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level [1.1999555634662635]
  LiDAR object detection algorithms based on neural networks for autonomous driving require large amounts of data for training, validation, and testing. We show that using simulated data for the training of neural networks leads to a domain shift of training and testing data due to differences in scenes, scenarios, and distributions.
  arXiv  Detail & Related papers  (2023-03-03T12:52:01Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• DSSIM: a structural similarity index for floating-point data [68.8204255655161]
  We propose an alternative to the popular SSIM that can be applied directly to the floating point data, which we refer to as the Data SSIM (DSSIM) While we demonstrate the usefulness of the DSSIM in the context of evaluating differences due to lossy compression on large volumes of simulation data, the DSSIM may prove useful for many other applications involving simulation or image data.
  arXiv  Detail & Related papers  (2022-02-05T19:18:33Z)
• Querying Labelled Data with Scenario Programs for Sim-to-Real Validation [5.8720142291102135]
  A fundamental question remains: are AV failure scenarios identified in simulation meaningful in reality, i.e., are they reproducible on the real system? An approach to validate simulated failure scenarios is to identify instances of the scenario in a corpus of real data, and check if the failure persists on the real data. We propose a formal definition of what it means for a labelled data item to match an abstract scenario, encoded as a scenario program using the SCENIC probabilistic programming language.
  arXiv  Detail & Related papers  (2021-12-01T01:04:13Z)
• Robust Event Classification Using Imperfect Real-world PMU Data [58.26737360525643]
  We study robust event classification using imperfect real-world phasor measurement unit (PMU) data. We develop a novel machine learning framework for training robust event classifiers.
  arXiv  Detail & Related papers  (2021-10-19T17:41:43Z)
• Deep Transformer Networks for Time Series Classification: The NPP Safety Case [59.20947681019466]
  An advanced temporal neural network referred to as the Transformer is used within a supervised learning fashion to model the time-dependent NPP simulation data. The Transformer can learn the characteristics of the sequential data and yield promising performance with approximately 99% classification accuracy on the testing dataset.
  arXiv  Detail & Related papers  (2021-04-09T14:26:25Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.