Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification

• URL: http://arxiv.org/abs/2608.13092v1
• Date: Thu, 13 Aug 2026 10:58:47 GMT
• Title: Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification
• Abstract summary: We propose a unified framework with disc-temporal modeling and hierarchical multi-modal fusion for RE-VReID.<n>Our framework can learn robust and discriminative representations for RE-VReID.
• Score: 75.15027201505912
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RGB-Event Video Person Re-Identification (RE-VReID) aims to retrieve specific person across non-overlapping cameras with complementary RGB videos and event streams. However, existing methods often decouple spatial and temporal modeling, which limits their interaction. In addition, global-level RGB-Event fusion fails to fully exploit fine-grained discriminative cues. To address these issues, we propose Paths, a unified framework with spatio-temporal modeling and hierarchical multi-modal fusion for RE-VReID. Specifically, we first design a Memory-Augmented Backbone (MAB) to maintain modality-specific identity prototypes for stable intra-modal representation learning. Then, we propose a Prompt-aware Spatio-temporal Transformer (PST) to jointly model spatial and temporal cues within a unified Transformer. Finally, we introduce a Hierarchical Multi-modal Fusion (HMF) to integrate RGB and event features at global and local levels. With these modules, our framework can learn robust and discriminative representations for RE-VReID. Extensive experiments on three public RE-VReID benchmarks including EvReID, MARS and iLIDS-VID, demonstrate the effectiveness of our proposed method. The code is available at https://github.com/Reflection0427/Paths.

Related papers

• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• CMTFormer: Marrying Transformer with Hierarchical Information Interaction for RGB-Event Object Detection [17.487415888287]
  Event cameras capture sparse brightness changes with high temporal resolution and high dynamic range.<n>Previous multi-modal fusion techniques fail to handle the inherent heterogeneity between RGB frames and event streams.<n>We propose a Cross-Modal information inTeraction transFormer, which hierarchically integrates RGB and event information.
  arXiv  Detail & Related papers  (2026-06-28T00:49:05Z)
• HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection [75.406055413928]
  We propose a novel prompt-driven segment anything model (HyPSAM) for RGB-T SOD.<n> DFNet employs dynamic convolution and multi-branch decoding to facilitate adaptive cross-modality interaction.<n>Experiments on three public datasets demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-23T07:32:11Z)
• CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework [30.734382771657312]
  We propose a novel CM3AE pre-training framework for the RGB-Event perception.<n>This framework accepts multi-modalities/views of data as input, including RGB images, event images, and event voxels.<n>We construct a large-scale dataset containing 2,535,759 RGB-Event data pairs for the pre-training.
  arXiv  Detail & Related papers  (2025-04-17T01:49:46Z)
• Multi-modal Multi-platform Person Re-Identification: Benchmark and Method [58.59888754340054]
  MP-ReID is a novel dataset designed specifically for multi-modality and multi-platform ReID.<n>This benchmark compiles data from 1,930 identities across diverse modalities, including RGB, infrared, and thermal imaging.<n>We introduce Uni-Prompt ReID, a framework with specific-designed prompts, tailored for cross-modality and cross-platform scenarios.
  arXiv  Detail & Related papers  (2025-03-21T12:27:49Z)
• Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification [46.49866514866999]
  We primarily study the video-based cross-modal person Re-ID method. We prove that with the increase of frames in a tracklet, the performance does meet more enhancement. A novel method is proposed, which projects two modalities to a modal-invariant subspace.
  arXiv  Detail & Related papers  (2022-08-04T04:43:52Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.