Zero-Label Driving Scenario Complexity Detection via Joint Embedding Predictive Architecture

• URL: http://arxiv.org/abs/2606.28383v1
• Date: Sun, 21 Jun 2026 05:26:58 GMT
• Title: Zero-Label Driving Scenario Complexity Detection via Joint Embedding Predictive Architecture
• Abstract summary: We ask whether a model can discover scenario complexity on its own, with no labels at any stage.<n>We train a minimal Joint Embedding Predictive Architecture (JEPA) on structured agent state data from the nuPlan mini dataset.<n>Without access to any ground-truth labels during training or evaluation setup, the model assigns significantly higher scores to scenarios involving unprotected turns, crosswalk interactions, and pedestrian proximity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying complex and safety-critical driving scenarios in large unlabelled datasets is an important but expensive problem. Existing approaches rely on human annotators, supervised classifiers, or carefully engineered rule sets, all of which require substantial prior knowledge about what constitutes a difficult scenario. We ask whether a model can discover scenario complexity on its own, with no labels at any stage. We train a minimal Joint Embedding Predictive Architecture (JEPA) on structured agent state data from the nuPlan mini dataset and use the temporal prediction error as a zero-shot complexity score. Without access to any ground-truth labels during training or evaluation setup, the model assigns significantly higher scores to scenarios involving unprotected turns, crosswalk interactions, and pedestrian proximity, and significantly lower scores to lane-following and stationary-traffic scenarios. We validate this finding through four ablation experiments that isolate the source of the signal, and through a downstream anomaly detection evaluation that achieves Average Precision of 0.512 against a 0.436 chance baseline. The results show that temporal prediction error in a self-supervised latent world model is a practical proxy for driving scenario complexity.

Related papers

• A Pedestrian-Vehicle Interaction Benchmark and Annotation Framework for Unstructured Scenes via Uncalibrated Cameras [14.16993152950793]
  We propose a dataset based on video data from uncalibrated surveillance cameras and present PINNS (Pedestrian-vehicle Interaction dataset from uNcalibrated cameras in uNstructured Scenes)<n>The dataset covers multiple countries and regions, includes diverse typical traffic scenarios, and considers variations in seasons, lighting conditions, and weather.<n>We hope our framework and dataset will facilitate research on trajectory prediction and autonomous driving in complex mixed traffic scenarios.
  arXiv  Detail & Related papers  (2026-05-25T15:26:18Z)
• Learning Shortest Paths When Data is Scarce [3.3012620893449465]
  We study a shortest-path problem in which a planner has access to abundant synthetic samples, limited real-world observations, and an edge-similarity capturing expected behavioral similarity across links.<n>We model the simulator-to-reality discrepancy as an unknown, edge-specific bias that varies smoothly over the similarity graph, and estimate it using Laplacian-regularized least squares.<n>For cold-start settings without initial real data, we develop a bias-aware active learning algorithm that adaptively selects edges to measure until a prescribed accuracy is met.
  arXiv  Detail & Related papers  (2026-01-07T06:19:04Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction [62.385417528148224]
  We present OmniTraj, a Transformer-based model pre-trained on a large-scale, heterogeneous dataset.<n>Experiments show that explicitly conditioning on the frame rate enables OmniTraj to achieve state-of-the-art zero-shot transfer performance.
  arXiv  Detail & Related papers  (2025-07-31T15:37:09Z)
• Inverse Reinforcement Learning for Minimum-Exposure Paths in Spatiotemporally Varying Scalar Fields [49.1574468325115]
  We consider a problem of synthesizing datasets of minimum exposure paths that resemble a training dataset of such paths.<n>The main contribution of this paper is an inverse reinforcement learning (IRL) model to solve this problem.<n>We find that the proposed IRL model provides excellent performance in synthesizing paths from initial conditions not seen in the training dataset.
  arXiv  Detail & Related papers  (2025-03-09T13:30:11Z)
• SSTP: Efficient Sample Selection for Trajectory Prediction [38.92588125424176]
  Training advanced trajectory prediction models on existing large-scale datasets is time-consuming and computationally expensive.<n>We propose the SSTP framework, which constructs a compact yet density-balanced dataset tailored to trajectory prediction.<n> Experiments show that SSTP achieves comparable performance to full-dataset training using only half the data.
  arXiv  Detail & Related papers  (2024-09-25T22:00:11Z)
• JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds [79.00975648564483]
  Trajectory forecasting models, employed in fields such as robotics, autonomous vehicles, and navigation, face challenges in real-world scenarios. This dataset provides comprehensive data, including the locations of all agents, scene images, and point clouds, all from the robot's perspective. The objective is to predict the future positions of agents relative to the robot using raw sensory input data.
  arXiv  Detail & Related papers  (2023-11-05T18:59:31Z)
• Graph Convolutional Networks for Complex Traffic Scenario Classification [0.7919810878571297]
  A scenario-based testing approach can reduce the time required to obtain statistically significant evidence of the safety of Automated Driving Systems. Most methods on scenario classification do not work for complex scenarios with diverse environments. We propose a method for complex traffic scenario classification that is able to model the interaction of a vehicle with the environment.
  arXiv  Detail & Related papers  (2023-10-26T20:51:24Z)
• Toward Unsupervised Test Scenario Extraction for Automated Driving Systems from Urban Naturalistic Road Traffic Data [0.0]
  The presented method deploys an unsupervised machine learning pipeline to extract scenarios from road traffic data. It is evaluated for naturalistic road traffic data at urban intersections from the inD and the Silicon Valley Intersections datasets. Using hierarchical clustering the results show both a jump in overall accuracy of around 20% when moving from 4 to 5 clusters and a saturation effect starting at 41 clusters with an overall accuracy of 84%.
  arXiv  Detail & Related papers  (2022-02-14T10:55:14Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.