TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding

• URL: http://arxiv.org/abs/2608.25935v1
• Date: Wed, 26 Aug 2026 15:50:38 GMT
• Title: TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding
• Abstract summary: TAU-Agent is an agentic retrieval-augmented framework for traffic anomaly understanding.<n>Given a task query, a central retrieval agent orchestrates two visual perception tools, to retrieve and select query-relevant evidence.<n>We evaluate TAU-Agent on both the in-domain and the out-of-domain benchmarks from the AI City Challenge 2026.
• Score: 16.4778063596097
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic Anomaly Understanding (TAU) requires models and systems to detect, reason about, and explain anomalous events in transportation videos. To address this challenge, we propose TAU-Agent, an agentic retrieval-augmented framework for traffic anomaly understanding. Given a task query, a central retrieval agent orchestrates two visual perception tools, namely a Video Captioning Tool and an Open-Vocabulary Tracking Tool, to retrieve and select query-relevant evidence, including captions, temporal intervals, and object trajectories. The selected evidence, together with sampled video frames and the input query, is provided to a supervised fine-tuned vision-language model for final reasoning and answer generation. We evaluate TAU-Agent on both the in-domain and the out-of-domain benchmarks from the AI City Challenge 2026. TAU-Agent achieves scores of 0.6779 on Track 3, 0.3998 on Track 7, and 67.9275 on Track 8, ranking second, twelfth, and fifth, respectively. Code is available at: https://github.com/siri-rouser/TAU-Agent.

Related papers

• WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents [53.426679066400844]
  We introduce WeAgent-Harness, a multimodal agentic harness that supports native text-vision interaction and runtime recovery.<n>We develop WeAgent-MMSearch, an integrated system spanning data construction, agentic post-training, and multimodal rollout.<n>We also introduce VisTarget-Bench, a 150-task human-verified benchmark that pairs each question with a held-out target image.
  arXiv  Detail & Related papers  (2026-08-28T08:28:43Z)
• UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations [87.58552218939225]
  We introduce UniTraffic-Agent, the MR-CAS solution for Track3 of the 10th AI City Challenge.<n>UniTraffic-Agent follows an observe--reason--act-verify workflow that samples timestamped visual evidence.<n>On the official Public leaderboards, MR-CAS ranks 16th on TAR with a score of 0.5780, 2nd on FETV with 0.4884, and 4th on PSI-VQA with 64.4161.
  arXiv  Detail & Related papers  (2026-08-13T10:00:27Z)
• ARGOS: Who, Where, and When in Agentic Multi-Camera Person Search [49.61286310968402]
  We introduce ARGOS, the first benchmark and framework that reformulates multi-camera person search as an interactive reasoning problem.<n>An ARGOS agent receives a vague witness statement and must decide what to ask, when to invoke spatial or temporal tools, and how to interpret ambiguous responses.<n>The benchmark comprises 2,691 tasks across 14 real-world scenarios in three progressive tracks: semantic perception (Who), spatial reasoning (Where), and temporal reasoning (When)
  arXiv  Detail & Related papers  (2026-04-14T14:06:19Z)
• SGTA: Scene-Graph Based Multi-Modal Traffic Agent for Video Understanding [48.47968852253859]
  Scene-Graph Based Multi-Modal Traffic Agent (SGTA) is a modular framework for traffic video understanding.<n>It constructs a traffic scene graph from roadside videos using detection, tracking, and lane extraction, followed by tool-based reasoning over both symbolic graph queries and visual inputs.
  arXiv  Detail & Related papers  (2026-04-04T12:01:49Z)
• RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocations [52.752467948588816]
  We propose RSAgent, an agentic Multimodal Large Language Model (MLLM) which interleaves reasoning and action for segmentation via multi-turn tool invocations.<n> RSAgent queries a segmentation toolbox, observes visual feedback, and revises its spatial hypothesis using historical observations to re-localize targets and iteratively refine masks.<n>Extensive experiments show that RSAgent achieves a zero-shot performance of 66.5% gIoU on ReasonSeg test, improving over Seg-Zero-7B by 9%, and reaches 81.5% cIoU on RefCOCOg, demonstrating state-of-the-art performance
  arXiv  Detail & Related papers  (2025-12-30T06:50:11Z)
• AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering [51.07491603393163]
  tAgent is a framework that formulates multi-agent QA as a knowledge-graph-guided routing problem supervised by empirical performance signals.<n>By leveraging soft supervision and weighted aggregation of agent outputs, Agent learns principled collaboration schemes that capture the complementary strengths of diverse agents.
  arXiv  Detail & Related papers  (2025-10-06T23:20:49Z)
• InterAct-Video: Reasoning-Rich Video QA for Urban Traffic [21.849445040376537]
  Deep learning has advanced video-based traffic monitoring through question answering (VideoQA) models.<n>Existing VideoQA models struggle with the complexity of real-world traffic scenes.<n>InterAct VideoQA is a curated dataset designed to benchmark and enhance VideoQA models for traffic monitoring tasks.
  arXiv  Detail & Related papers  (2025-07-19T20:30:43Z)
• ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association [15.161640917854363]
  We introduce ADA-Track++, a novel end-to-end framework for 3D MOT from multi-view cameras.<n>We introduce a learnable data association module based on edge-augmented cross-attention.<n>We integrate this association module into the decoder layer of a DETR-based 3D detector.
  arXiv  Detail & Related papers  (2024-05-14T19:02:33Z)
• End-to-end Tracking with a Multi-query Transformer [96.13468602635082]
  Multiple-object tracking (MOT) is a challenging task that requires simultaneous reasoning about location, appearance, and identity of the objects in the scene over time. Our aim in this paper is to move beyond tracking-by-detection approaches, to class-agnostic tracking that performs well also for unknown object classes.
  arXiv  Detail & Related papers  (2022-10-26T10:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.