UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations

• URL: http://arxiv.org/abs/2608.13031v1
• Date: Thu, 13 Aug 2026 10:00:27 GMT
• Title: UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations
• Abstract summary: We introduce UniTraffic-Agent, the MR-CAS solution for Track3 of the 10th AI City Challenge.<n>UniTraffic-Agent follows an observe--reason--act-verify workflow that samples timestamped visual evidence.<n>On the official Public leaderboards, MR-CAS ranks 16th on TAR with a score of 0.5780, 2nd on FETV with 0.4884, and 4th on PSI-VQA with 64.4161.
• Score: 87.58552218939225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traffic video understanding has become an important problem in intelligent transportation, as road videos provide direct evidence for accidents, violations, and interactions between vehicles and vulnerable road users. A useful system should explain how a traffic event develops, why it happens, and when the relevant interaction occurs, yet this remains difficult for multimodal large language models (MLLMs) because traffic videos contain sparse events and varied viewpoints. We introduce UniTraffic-Agent, the MR-CAS solution for Track~3 of the 10th AI City Challenge, which includes Traffic Anomaly Reasoning (TAR) and two out-of-domain evaluations: FETV for fisheye traffic events and PSI-VQA for pedestrian intention reasoning. UniTraffic-Agent follows an observe--reason--act--verify workflow that samples timestamped visual evidence, reasons over all questions from the same clip in one request, and converts responses through task-specific action adapters. On the official Public leaderboards, MR-CAS ranks 16th on TAR with a score of 0.5780, 2nd on FETV with 0.4884, and 4th on PSI-VQA with 64.4161. The code is available at https://github.com/Roclp/UniTraffic-Agent.

Related papers

• TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding [16.4778063596097]
  TAU-Agent is an agentic retrieval-augmented framework for traffic anomaly understanding.<n>Given a task query, a central retrieval agent orchestrates two visual perception tools, to retrieve and select query-relevant evidence.<n>We evaluate TAU-Agent on both the in-domain and the out-of-domain benchmarks from the AI City Challenge 2026.
  arXiv  Detail & Related papers  (2026-08-26T15:50:38Z)
• OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning [47.85568621735744]
  We introduce OmniTraffic, a controllable pipeline generation and benchmark for traffic reasoning.<n>It is built around 12 real-world intersections and reconstructed by surveillance footage from two countries.<n>It defines a three-level task hierarchy spanning scene perception, multi-view and temporal reasoning, and decision support.
  arXiv  Detail & Related papers  (2026-06-14T11:16:53Z)
• InterAct-Video: Reasoning-Rich Video QA for Urban Traffic [21.849445040376537]
  Deep learning has advanced video-based traffic monitoring through question answering (VideoQA) models.<n>Existing VideoQA models struggle with the complexity of real-world traffic scenes.<n>InterAct VideoQA is a curated dataset designed to benchmark and enhance VideoQA models for traffic monitoring tasks.
  arXiv  Detail & Related papers  (2025-07-19T20:30:43Z)
• Towards Intelligent Transportation with Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework [62.47416496137193]
  We propose a surveillance video assisted federated digital twin (SV-FDT) framework to empower ITSs with pedestrians and vehicles in-the-loop.<n>The architecture consists of three layers: (i) the end layer, which collects traffic surveillance videos from multiple sources; (ii) the edge layer, responsible for semantic segmentation-based visual understanding, twin agent-based interaction modeling, and local digital twin system (LDTS) creation in local regions; and (iii) the cloud layer, which integrates LDTSs across different regions to construct a global DT model in realtime.
  arXiv  Detail & Related papers  (2025-03-06T07:36:06Z)
• Eyes on the Road: State-of-the-Art Video Question Answering Models Assessment for Traffic Monitoring Tasks [0.0]
  This study evaluates state-of-the-art VideoQA models using non-benchmark synthetic and real-world traffic sequences.<n>VideoLLaMA-2 advances with 57% accuracy, particularly in compositional reasoning and consistent answers.<n>These findings underscore VideoQA's potential in traffic monitoring but also emphasize the need for improvements in multi-object tracking, temporal reasoning, and compositional capabilities.
  arXiv  Detail & Related papers  (2024-12-02T05:15:32Z)
• TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioning [0.0]
  We present TrafficVLM, a novel multi-modal dense video captioning model for vehicle ego camera view. Our solution achieved outstanding results in Track 2 of the AI City Challenge 2024, ranking us third in the challenge standings.
  arXiv  Detail & Related papers  (2024-04-14T14:51:44Z)
• Traffic-Domain Video Question Answering with Automatic Captioning [69.98381847388553]
  Video Question Answering (VidQA) exhibits remarkable potential in facilitating advanced machine reasoning capabilities. We present a novel approach termed Traffic-domain Video Question Answering with Automatic Captioning (TRIVIA), which serves as a weak-supervision technique for infusing traffic-domain knowledge into large video-language models.
  arXiv  Detail & Related papers  (2023-07-18T20:56:41Z)
• OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping [84.65114565766596]
  We present OpenLane-V2, the first dataset on topology reasoning for traffic scene structure. OpenLane-V2 consists of 2,000 annotated road scenes that describe traffic elements and their correlation to the lanes. We evaluate various state-of-the-art methods, and present their quantitative and qualitative results on OpenLane-V2 to indicate future avenues for investigating topology reasoning in traffic scenes.
  arXiv  Detail & Related papers  (2023-04-20T16:31:22Z)
• ROCO: A Roundabout Traffic Conflict Dataset [65.55451440776098]
  We introduce and analyze ROCO - a real-world roundabout traffic conflict dataset. The data is collected at a two-lane roundabout at the intersection of State St. and W. Ellsworth Rd. in Ann Arbor, Michigan. In total 557 traffic conflicts and 17 traffic crashes are collected from August 2021 to October 2021.
  arXiv  Detail & Related papers  (2023-03-01T15:09:45Z)
• TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events [13.46045177335564]
  We create a novel dataset, TrafficQA (Traffic Question Answering), based on the collected 10,080 in-the-wild videos and annotated 62,535 QA pairs. We propose 6 challenging reasoning tasks corresponding to various traffic scenarios, so as to evaluate the reasoning capability over different kinds of complex yet practical traffic events. We also propose Eclipse, a novel Efficient glimpse network via dynamic inference, in order to achieve computation-efficient and reliable video reasoning.
  arXiv  Detail & Related papers  (2021-03-29T12:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.