MSAM: Multi-Semantic Adaptive Mining for Cross-Modal Drone Video-Text Retrieval

• URL: http://arxiv.org/abs/2510.15470v1
• Date: Fri, 17 Oct 2025 09:26:28 GMT
• Title: MSAM: Multi-Semantic Adaptive Mining for Cross-Modal Drone Video-Text Retrieval
• Abstract summary: Drone videos feature overhead perspectives, strong structural homogeneity, and diverse semantic expressions of target combinations.<n>We propose a novel approach called Multi-Semantic Adaptive Mining (MSAM)<n>MSAM incorporates dynamic changes between frames and extracts rich semantic information from specific scene regions.
• Score: 10.526794142255492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advancement of drone technology, the volume of video data increases rapidly, creating an urgent need for efficient semantic retrieval. We are the first to systematically propose and study the drone video-text retrieval (DVTR) task. Drone videos feature overhead perspectives, strong structural homogeneity, and diverse semantic expressions of target combinations, which challenge existing cross-modal methods designed for ground-level views in effectively modeling their characteristics. Therefore, dedicated retrieval mechanisms tailored for drone scenarios are necessary. To address this issue, we propose a novel approach called Multi-Semantic Adaptive Mining (MSAM). MSAM introduces a multi-semantic adaptive learning mechanism, which incorporates dynamic changes between frames and extracts rich semantic information from specific scene regions, thereby enhancing the deep understanding and reasoning of drone video content. This method relies on fine-grained interactions between words and drone video frames, integrating an adaptive semantic construction module, a distribution-driven semantic learning term and a diversity semantic term to deepen the interaction between text and drone video modalities and improve the robustness of feature representation. To reduce the interference of complex backgrounds in drone videos, we introduce a cross-modal interactive feature fusion pooling mechanism that focuses on feature extraction and matching in target regions, minimizing noise effects. Extensive experiments on two self-constructed drone video-text datasets show that MSAM outperforms other existing methods in the drone video-text retrieval task. The source code and dataset will be made publicly available.

Related papers

• AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignment [32.90190674554767]
  AgentHOI is a text-driven HOI video generation that bridges the gap between high-level textual intent and physical execution.<n>We introduce an implicit text-motion alignment strategy that distills text-to-motion priors into the video diffusion model.<n>Experiments show that AgentHOI significantly improves interaction naturalness, object appearance preservation, and adherence to complex textual instructions.
  arXiv  Detail & Related papers  (2026-07-24T12:17:57Z)
• TCMA: Text-Conditioned Multi-granularity Alignment for Drone Cross-Modal Text-Video Retrieval [5.527227553079524]
  Unmanned aerial vehicles (UAVs) have become powerful platforms for real-time, high-resolution data collection.<n> Efficient retrieval of relevant content from these videos is crucial for applications in urban management, emergency response, security, and disaster relief.<n>We construct the Drone Video-Text Match dataset (DVTMD), which contains 2,864 videos and 14,320 fine-grained, semantically diverse captions.
  arXiv  Detail & Related papers  (2025-10-11T11:38:01Z)
• BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration [56.98981194478512]
  We propose a unified framework that handles a broad range of subject-to-video scenarios.<n>We introduce an MLLM-DiT framework in which a pretrained multimodal large language model performs deep cross-modal reasoning to ground entities.<n>Experiments on the OpenS2V benchmark demonstrate that our method achieves superior performance across subject consistency, naturalness, and text relevance in generated videos.
  arXiv  Detail & Related papers  (2025-10-01T02:41:11Z)
• FitPro: A Zero-Shot Framework for Interactive Text-based Pedestrian Retrieval in Open World [13.089848592467675]
  FitPro is an open-world interactive TPR framework with enhanced semantic comprehension and cross-scene adaptability.<n> FitPro has three innovative components: Feature Contrastive Decoding (FCD), Incremental Semantic Mining (ISM), and Query-aware Hierarchical Retrieval (QHR)
  arXiv  Detail & Related papers  (2025-09-20T12:55:18Z)
• VideoMolmo: Spatio-Temporal Grounding Meets Pointing [66.19964563104385]
  VideoMolmo is a model tailored for fine-grained pointing of video sequences.<n>A novel temporal mask fusion employs SAM2 for bidirectional point propagation.<n>To evaluate the generalization of VideoMolmo, we introduce VPoMolS-temporal, a challenging out-of-distribution benchmark spanning five real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-05T17:59:29Z)
• Prototypes are Balanced Units for Efficient and Effective Partially Relevant Video Retrieval [23.75587275795415]
  We propose a prototypical PRVR framework that encodes diverse contexts within a video into a fixed number of prototypes.<n>To keep the prototypes searchable via text queries while accurately encoding video contexts, we implement cross- and uni-modal reconstruction tasks.
  arXiv  Detail & Related papers  (2025-04-17T15:43:29Z)
• Redundancy-aware Transformer for Video Question Answering [71.98116071679065]
  We propose a novel transformer-based architecture, that aims to model VideoQA in a redundancy-aware manner. To address the neighboring-frame redundancy, we introduce a video encoder structure that emphasizes the object-level change in neighboring frames. As for the cross-modal redundancy, we equip our fusion module with a novel adaptive sampling, which explicitly differentiates the vision-language interactions.
  arXiv  Detail & Related papers  (2023-08-07T03:16:24Z)
• Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation [56.41614987789537]
  Text-based video segmentation aims to segment the target object in a video based on a describing sentence. We propose a method to fuse and align appearance, motion, and linguistic features to achieve accurate segmentation.
  arXiv  Detail & Related papers  (2022-04-06T02:42:33Z)
• Align and Prompt: Video-and-Language Pre-training with Entity Prompts [111.23364631136339]
  Video-and-language pre-training has shown promising improvements on various downstream tasks. We propose Align and Prompt: an efficient and effective video-and-language pre-training framework with better cross-modal alignment. Our code and pre-trained models will be released.
  arXiv  Detail & Related papers  (2021-12-17T15:55:53Z)
• Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers [89.00926092864368]
  We present a semantics-controlled multi-modal shuffled Transformer reasoning framework for the audio-visual scene aware dialog task. We also present a novel dynamic scene graph representation learning pipeline that consists of an intra-frame reasoning layer producing-semantic graph representations for every frame. Our results demonstrate state-of-the-art performances on all evaluation metrics.
  arXiv  Detail & Related papers  (2020-07-08T02:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.