Predictive audio representations for early detection and tracking of hidden dynamic objects

• URL: http://arxiv.org/abs/2609.13595v1
• Date: Fri, 11 Sep 2026 23:05:23 GMT
• Title: Predictive audio representations for early detection and tracking of hidden dynamic objects
• Abstract summary: Occluded traffic agents challenge detection systems as they might become visible too late.<n>Previous works proved that auditory perception, being omnidirectional and not constrained by a field-of-view, provides fundamental cues for early spotting of different road users.<n>In this work, we propose a multi-task system which, simultaneously, estimates the number of vehicles present, their type, and their direction of arrival.
• Score: 1.7289819674602296
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Predicting potential dangers is core to safety. Forecasting the presence of other traffic agents is core to danger prediction. Occluded traffic agents challenge detection systems as they might become visible too late, leaving the autonomous vehicle too little time to identify, plan and act accordingly in a robust and safe way. Previous works proved that auditory perception, being omnidirectional and not constrained by a field-of-view, provides fundamental cues for early spotting of different road users, even when hidden by other vehicles or infrastructures. Yet, those contributions deal with scenarios with only one vehicle present, and they either identify the type of vehicle or estimate its direction of arrival. In this work, we move forward, and propose a multi-task system which, simultaneously, estimates the number of vehicles present, their type, and their direction of arrival. Our methodological contribution is a two-stage pipeline: a self-supervised pre-training stage inspired by the Joint- Embedding Predictive Architecture (JEPA) applied directly to multichannel raw waveforms, followed by supervised multi-task fine-tuning with a bidirectional LSTM and three classification heads. The pre-training stage trains the encoder without labels, pushing it to predict the latent representation of a future audio segment from its past context. To train and test our framework, since no suitable dataset was publicly available, we collected an ad-hoc one covering Non-Line-Of-Sight scenarios with multiple traffic agents simultaneously operating. Experimental evaluation shows that our method outperforms the state of the art; it also proves that our design choice allows the model to learn robust representations, which can be transferred to an unseen driving scenario, maintaining reasonable and stable performance.

Related papers

• QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving [33.609780917199394]
  Self-driving vehicles must understand its environment to determine appropriate action. Traditional systems rely on object detection to find agents in the scene. We present a unified, interpretable, and efficient autonomy framework that moves away from cascading modules that first perceive occupancy relevant-temporal autonomy.
  arXiv  Detail & Related papers  (2024-04-01T21:11:43Z)
• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding [51.8579160500354]
  We propose an unsupervised way to predict self-driving attention by uncertainty modeling and driving knowledge integration. Results show equivalent or even more impressive performance compared to fully-supervised state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-03-17T00:28:33Z)
• Unsupervised Driving Event Discovery Based on Vehicle CAN-data [62.997667081978825]
  This work presents a simultaneous clustering and segmentation approach for vehicle CAN-data that identifies common driving events in an unsupervised manner. We evaluate our approach with a dataset of real Tesla Model 3 vehicle CAN-data and a two-hour driving session that we annotated with different driving events.
  arXiv  Detail & Related papers  (2023-01-12T13:10:47Z)
• Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving [104.32241082170044]
  We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving. Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map. Our approach is the first one that can predict the existence of unseen vehicles in most cases.
  arXiv  Detail & Related papers  (2021-09-03T13:33:33Z)
• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving [139.33800431159446]
  LookOut is an approach to jointly perceive the environment and predict a diverse set of futures from sensor data. We show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset.
  arXiv  Detail & Related papers  (2021-01-16T23:19:22Z)
• Two-Stream Networks for Lane-Change Prediction of Surrounding Vehicles [8.828423067460644]
  In highway scenarios, an alert human driver will typically anticipate early cut-in and cut-out maneuvers surrounding vehicles using only visual cues. To deal with lane-change recognition and prediction of surrounding vehicles, we pose the problem as an action recognition/prediction problem by stacking visual cues from video cameras. Two video action recognition approaches are analyzed: two-stream convolutional networks and multiplier networks.
  arXiv  Detail & Related papers  (2020-08-25T07:59:15Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)
• Probabilistic Multi-modal Trajectory Prediction with Lane Attention for Autonomous Vehicles [10.485790589381704]
  Trajectory prediction is crucial for autonomous vehicles. We propose a novel instance-aware representation for lane representation. We show that the proposed lane representation together with the lane attention module can be integrated into the widely used encoder-decoder framework.
  arXiv  Detail & Related papers  (2020-07-06T07:57:23Z)
• Physically constrained short-term vehicle trajectory forecasting with naive semantic maps [6.85316573653194]
  We propose a model that learns to extract relevant road features from semantic maps as well as general motion of agents. We show that our model is not only capable of anticipating future motion whilst taking into consideration road boundaries, but can also effectively and precisely predict trajectories for a longer time horizon than initially trained for.
  arXiv  Detail & Related papers  (2020-06-09T09:52:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.