Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval

• URL: http://arxiv.org/abs/2608.12843v1
• Date: Thu, 13 Aug 2026 05:28:50 GMT
• Title: Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval
• Abstract summary: This paper presents the GENAI4E team's solution to AI City Challenge 2026 Track 4.<n>Our framework builds upon a strong retrieval backbone and integrates heterogeneous vision-language embedding models.<n>On the official Pedestrian Anomaly Behavior benchmark, our approach achieves 90.92% mAP.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-based person anomaly retrieval aims to retrieve pedestrians exhibiting anomalous behaviors from a large image gallery using natural language descriptions. Compared with conventional text-based person retrieval, this task requires fine-grained reasoning over pedestrian appearance, behaviors, object interactions, and scene context, making robust cross-modal matching significantly more challenging. This paper presents the GENAI4E team's solution to AI City Challenge 2026 Track 4. Our framework builds upon a strong retrieval backbone and progressively integrates heterogeneous vision-language embedding models through score alignment and iterative ensemble fusion, followed by disagreement-aware VLM reranking for ambiguous queries. On the official Pedestrian Anomaly Behavior (PAB) benchmark, our approach achieves 90.92% mAP, 85.13% Recall@1, 97.72% Recall@5, and 98.68% Recall@10, demonstrating the effectiveness of combining complementary vision-language representations with selective multimodal reasoning for large-scale text-based person anomaly retrieval.

Related papers

• Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search [10.617164505348283]
  Text-based person anomaly search requires distinguishing individuals based on fine-grained, context-dependent behaviors rather than mere appearance.<n>Existing methods struggle to capture these context-conditioned actions.<n>We propose a unified three-stage coarse-to-fine framework that combines action-aligned retrieval with pairwise multimodal reranking.
  arXiv  Detail & Related papers  (2026-08-24T17:04:34Z)
• FaLCon: Facet-Anchored Retrieval with Late Consensus for Sim2Real Text-Based Person Anomaly Search [0.8609344172589112]
  Text-based person anomaly search requires retrieving real-world pedestrian images from detailed natural-language descriptions.<n>This Sim2Real setting is particularly challenging because visually similar candidates may differ only in subtle actions, object interactions, or appearance attributes.<n>We propose an anchor-constrained coarse-to-fine retrieval framework that combines global semantic matching with fine-grained verification.
  arXiv  Detail & Related papers  (2026-08-10T11:42:52Z)
• Team RAS in 11th ABAW Competition: Multimodal Ambivalence Recognition Approach [52.92973028971813]
  We present a single text-centered multimodal approach for video-level ambivalence and hesitancy recognition.<n>The proposed approach combines linguistic, acoustic, facial, and scene features using text-centered multimodal fusion model.<n>Experiments on the Behavioural Ambivalence/Hesitancy (BAH) corpus confirm that text is the strongest unimodal modality.
  arXiv  Detail & Related papers  (2026-07-16T08:04:23Z)
• DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories [52.57197752244638]
  We introduce DeepImageSearch, a novel agentic paradigm that reformulates image retrieval as an autonomous exploration task.<n>Models must plan and perform multi-step reasoning over raw visual histories to locate targets based on implicit contextual cues.<n>We construct DISBench, a challenging benchmark built on interconnected visual data.
  arXiv  Detail & Related papers  (2026-02-11T12:51:10Z)
• Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning [66.52010873968383]
  We introduce a conversational agent that interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training.<n>The experimental results across four widely used conversational benchmarks demonstrate the effectiveness of our methods.
  arXiv  Detail & Related papers  (2026-01-19T14:55:54Z)
• Towards Context-aware Reasoning-enhanced Generative Searching in E-commerce [61.03081096959132]
  We propose a context-aware reasoning-enhanced generative search framework for better textbfunderstanding the complicated context.<n>Our approach achieves superior performance compared with strong baselines, validating its effectiveness for search-based recommendation.
  arXiv  Detail & Related papers  (2025-10-19T16:46:11Z)
• Composed Object Retrieval: Object-level Retrieval via Composed Expressions [71.47650333199628]
  Composed Object Retrieval (COR) is a brand-new task that goes beyond image-level retrieval to achieve object-level precision.<n>We construct COR127K, the first large-scale COR benchmark that contains 127,166 retrieval triplets with various semantic transformations in 408 categories.<n>We also present CORE, a unified end-to-end model that integrates reference region encoding, adaptive visual-textual interaction, and region-level contrastive learning.
  arXiv  Detail & Related papers  (2025-08-06T13:11:40Z)
• ChatReID: Open-ended Interactive Person Retrieval via Hierarchical Progressive Tuning for Vision Language Models [49.09606704563898]
  Person re-identification is a crucial task in computer vision, aiming to recognize individuals across non-overlapping camera views.<n>We propose a novel framework ChatReID, that shifts the focus towards a text-side-dominated retrieval paradigm, enabling flexible and interactive re-identification.<n>We introduce a hierarchical progressive tuning strategy, which endows Re-ID ability through three stages of tuning, i.e., from person attribute understanding to fine-grained image retrieval and to multi-modal task reasoning.
  arXiv  Detail & Related papers  (2025-02-27T10:34:14Z)
• Efficient Vision Language Model Fine-tuning for Text-based Person Anomaly Search [20.695290280579858]
  This paper presents the HFUT-LMC team's solution to the WWW 2025 challenge on Text-based Person Anomaly Search (TPAS)<n>The primary objective of this challenge is to accurately identify pedestrians exhibiting either normal or abnormal behavior within a large library of pedestrian images.<n>We introduce the Similarity Coverage Analysis (SCA) strategy to address the recognition difficulty caused by similar text descriptions.
  arXiv  Detail & Related papers  (2025-02-05T14:45:09Z)
• Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search [25.907668574771705]
  We propose a new task, text-based person anomaly search, locating pedestrians engaged in both routine or anomalous activities via text.<n>To enable the training and evaluation of this new task, we construct a large-scale image-text Pedestrian Anomaly Behavior benchmark.<n>Experiments on the proposed benchmark show that synthetic training data facilitates the fine-grained behavior retrieval, and the proposed pose-aware method arrives at 84.93% recall@1 accuracy.
  arXiv  Detail & Related papers  (2024-11-26T09:50:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.