Towards Accurate and Robust Surveillance Roadside IVD via Trackletized Audio-Visual Reasoning

• URL: http://arxiv.org/abs/2606.22299v1
• Date: Sun, 21 Jun 2026 01:58:04 GMT
• Title: Towards Accurate and Robust Surveillance Roadside IVD via Trackletized Audio-Visual Reasoning
• Abstract summary: Idling Vehicle Detection (IVD) seeks to determine, at the final frame of a video clip, whether any vehicle is idling.<n>We introduce TAVR-IVD, an audio-visual framework guided by multi-object tracking.
• Score: 0.9799637101641149
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Idling Vehicle Detection (IVD) seeks to determine, at the final frame of a video clip, whether any vehicle is idling, meaning the vehicle is stationary with its engine running, using synchronized video from a remote surveillance camera and multichannel audio captured by spatially distributed wireless microphones along the roadside. Prior full-image, clip-level fusion approaches tend to overfit scene background and full-frame context, produce unstable temporal decisions, and lack an explicit spatial prior to align vehicles with microphones, which makes them brittle under domain shift and data inefficient. Instead, we introduce TAVR-IVD, an audio-visual framework guided by multi-object tracking. Our method detects vehicles, links detections into tracklets, and classifies each vehicle by operating on its tracklet. This design raises the effective signal-to-noise ratio, stabilizes temporal decisions through tracklets, enforces an explicit spatial prior to align vehicles with microphones, and adapts across domains with limited calibration annotations while remaining detector agnostic and efficient. To evaluate deployment robustness, we further curate two evaluation extensions, AVIVD-LT and AVIVD-M, covering inter-day and cross-site shifts.

Related papers

• OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning [47.85568621735744]
  We introduce OmniTraffic, a controllable pipeline generation and benchmark for traffic reasoning.<n>It is built around 12 real-world intersections and reconstructed by surveillance footage from two countries.<n>It defines a three-level task hierarchy spanning scene perception, multi-view and temporal reasoning, and decision support.
  arXiv  Detail & Related papers  (2026-06-14T11:16:53Z)
• Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis [53.67632483195509]
  We introduce StreetNVS, a video diffusion framework that conditions on all three signals through a Reference-Enhanced Camera Attention module.<n>We show capabilities of synthesizing coherent videos along extreme out-of-trajectory paths such as elevation, lane-shift, pullback, and rotation.
  arXiv  Detail & Related papers  (2026-06-01T02:37:56Z)
• SPOT!: Map-Guided LLM Agent for Unsupervised Multi-CCTV Dynamic Object Tracking [0.0]
  This paper proposes SPOT (Spatial Prediction Over Trajectories), a map-guided LLM agent capable of tracking vehicles even in blind spots of multi-CCTV environments without prior training.<n>It transforms the vehicle's position into the actual world coordinate system using the relative position and FOV information of objects observed in CCTV images.<n> Experimental results based on the CARLA simulator in a virtual city environment confirmed that the proposed method accurately predicts the next appearing CCTV even in blind spot sections.
  arXiv  Detail & Related papers  (2025-12-24T06:04:58Z)
• SAE-MCVT: A Real-Time and Scalable Multi-Camera Vehicle Tracking Framework Powered by Edge Computing [2.9754058024342473]
  We propose SAE-MCVT, the first scalable real-time MCVT framework.<n>We show that SAE-MCVT maintains real-time operation on 2K 15 FPS video streams and achieves an IDF1 score of 61.2.<n>This is the first scalable real-time MCVT framework suitable for city-scale deployment.
  arXiv  Detail & Related papers  (2025-11-17T20:55:14Z)
• Tracking the Unstable: Appearance-Guided Motion Modeling for Robust Multi-Object Tracking in UAV-Captured Videos [58.156141601478794]
  Multi-object tracking (UAVT) aims to track multiple objects while maintaining consistent identities across frames of a given video.<n>Existing methods typically model motion cues and appearance separately, overlooking their interplay and resulting in suboptimal tracking performance.<n>We propose AMOT, which exploits appearance and motion cues through two key components: an Appearance-Motion Consistency (AMC) matrix and a Motion-aware Track Continuation (MTC) module.
  arXiv  Detail & Related papers  (2025-08-03T12:06:47Z)
• HAVT-IVD: Heterogeneity-Aware Cross-Modal Network for Audio-Visual Surveillance: Idling Vehicles Detection With Multichannel Audio and Multiscale Visual Cues [1.0705399532413615]
  Idling vehicle detection (IVD) uses surveillance video and multichannel audio to localize and classify vehicles in pick-up zones.<n>IVD faces three challenges: (i) modality heterogeneity between visual cues and audio patterns; (ii) large box scale variation requiring multi-resolution detection; and (iii) training instability due to coupled detection heads.<n>We propose HAVT-IVD, a heterogeneous-aware network with a visual feature pyramid and decoupled heads.
  arXiv  Detail & Related papers  (2025-04-15T21:10:17Z)
• Towards Intelligent Transportation with Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework [62.47416496137193]
  We propose a surveillance video assisted federated digital twin (SV-FDT) framework to empower ITSs with pedestrians and vehicles in-the-loop.<n>The architecture consists of three layers: (i) the end layer, which collects traffic surveillance videos from multiple sources; (ii) the edge layer, responsible for semantic segmentation-based visual understanding, twin agent-based interaction modeling, and local digital twin system (LDTS) creation in local regions; and (iii) the cloud layer, which integrates LDTSs across different regions to construct a global DT model in realtime.
  arXiv  Detail & Related papers  (2025-03-06T07:36:06Z)
• Application of 2D Homography for High Resolution Traffic Data Collection using CCTV Cameras [9.946460710450319]
  This study implements a three-stage video analytics framework for extracting high-resolution traffic data from CCTV cameras. The key components of the framework include object recognition, perspective transformation, and vehicle trajectory reconstruction. The results of the study showed about +/- 4.5% error rate for directional traffic counts, less than 10% MSE for speed bias between camera estimates.
  arXiv  Detail & Related papers  (2024-01-14T07:33:14Z)
• Implicit Motion Handling for Video Camouflaged Object Detection [60.98467179649398]
  We propose a new video camouflaged object detection (VCOD) framework. It can exploit both short-term and long-term temporal consistency to detect camouflaged objects from video frames.
  arXiv  Detail & Related papers  (2022-03-14T17:55:41Z)
• Real Time Monocular Vehicle Velocity Estimation using Synthetic Data [78.85123603488664]
  We look at the problem of estimating the velocity of road vehicles from a camera mounted on a moving car. We propose a two-step approach where first an off-the-shelf tracker is used to extract vehicle bounding boxes and then a small neural network is used to regress the vehicle velocity.
  arXiv  Detail & Related papers  (2021-09-16T13:10:27Z)
• Online Clustering-based Multi-Camera Vehicle Tracking in Scenarios with overlapping FOVs [2.6365690297272617]
  Multi-Target Multi-Camera (MTMC) vehicle tracking is an essential task of visual traffic monitoring. We present a new low-latency online approach for MTMC tracking in scenarios with partially overlapping fields of view.
  arXiv  Detail & Related papers  (2021-02-08T09:55:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.