RGB-Event based Pedestrian Attribute Recognition: A Benchmark Dataset and An Asymmetric RWKV Fusion Framework

• URL: http://arxiv.org/abs/2504.10018v1
• Date: Mon, 14 Apr 2025 09:22:16 GMT
• Title: RGB-Event based Pedestrian Attribute Recognition: A Benchmark Dataset and An Asymmetric RWKV Fusion Framework
• Authors: Xiao Wang, Haiyang Wang, Shiao Wang, Qiang Chen, Jiandong Jin, Haoyu Song, Bo Jiang, Chenglong Li,
• Abstract summary: Existing pedestrian attribute recognition methods are generally developed based on RGB frame cameras.<n>We propose a novel multi-modal RGB-Event attribute recognition task by drawing inspiration from the advantages of event cameras in low-light, high-speed, and low-power consumption.<n>Specifically, we introduce the first large-scale multi-modal pedestrian attribute recognition dataset, termed EventPAR.
• Score: 20.19599141770658
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing pedestrian attribute recognition methods are generally developed based on RGB frame cameras. However, these approaches are constrained by the limitations of RGB cameras, such as sensitivity to lighting conditions and motion blur, which hinder their performance. Furthermore, current attribute recognition primarily focuses on analyzing pedestrians' external appearance and clothing, lacking an exploration of emotional dimensions. In this paper, we revisit these issues and propose a novel multi-modal RGB-Event attribute recognition task by drawing inspiration from the advantages of event cameras in low-light, high-speed, and low-power consumption. Specifically, we introduce the first large-scale multi-modal pedestrian attribute recognition dataset, termed EventPAR, comprising 100K paired RGB-Event samples that cover 50 attributes related to both appearance and six human emotions, diverse scenes, and various seasons. By retraining and evaluating mainstream PAR models on this dataset, we establish a comprehensive benchmark and provide a solid foundation for future research in terms of data and algorithmic baselines. In addition, we propose a novel RWKV-based multi-modal pedestrian attribute recognition framework, featuring an RWKV visual encoder and an asymmetric RWKV fusion module. Extensive experiments are conducted on our proposed dataset as well as two simulated datasets (MARS-Attribute and DukeMTMC-VID-Attribute), achieving state-of-the-art results. The source code and dataset will be released on https://github.com/Event-AHU/OpenPAR

Related papers

• Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset [65.76480665062363]
  Human Activity Recognition primarily relied on traditional RGB cameras to achieve high-performance activity recognition.<n>Challenges in real-world scenarios, such as insufficient lighting and rapid movements, inevitably degrade the performance of RGB cameras.<n>In this work, we rethink human activity recognition by combining the RGB and event cameras.
  arXiv  Detail & Related papers  (2025-04-08T09:14:24Z)
• Multi-modal Multi-platform Person Re-Identification: Benchmark and Method [58.59888754340054]
  MP-ReID is a novel dataset designed specifically for multi-modality and multi-platform ReID. This benchmark compiles data from 1,930 identities across diverse modalities, including RGB, infrared, and thermal imaging. We introduce Uni-Prompt ReID, a framework with specific-designed prompts, tailored for cross-modality and cross-platform scenarios.
  arXiv  Detail & Related papers  (2025-03-21T12:27:49Z)
• When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset [40.24765100535353]
  This paper introduces MMPedestron, a novel generalist model for multimodal perception. The proposed approach comprises a unified encoder for modal representation and fusion and a general head for pedestrian detection. With multi-modal joint training, our model achieves state-of-the-art performance on a wide range of pedestrian detection benchmarks.
  arXiv  Detail & Related papers  (2024-07-14T09:16:49Z)
• TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking [30.89375068036783]
  Existing approaches perform event feature extraction for RGB-E tracking using traditional appearance models. We propose an Event backbone (Pooler) to obtain a high-quality feature representation that is cognisant of the intrinsic characteristics of the event data. Our method significantly outperforms state-of-the-art trackers on two widely used RGB-E tracking datasets.
  arXiv  Detail & Related papers  (2024-05-08T12:19:08Z)
• Spatio-Temporal Side Tuning Pre-trained Foundation Models for Video-based Pedestrian Attribute Recognition [58.79807861739438]
  Existing pedestrian recognition (PAR) algorithms are mainly developed based on a static image. We propose to understand human attributes using video frames that can fully use temporal information.
  arXiv  Detail & Related papers  (2024-04-27T14:43:32Z)
• Segment Any Events via Weighted Adaptation of Pivotal Tokens [85.39087004253163]
  This paper focuses on the nuanced challenge of tailoring the Segment Anything Models (SAMs) for integration with event data. We introduce a multi-scale feature distillation methodology to optimize the alignment of token embeddings originating from event data with their RGB image counterparts.
  arXiv  Detail & Related papers  (2023-12-24T12:47:08Z)
• Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera [8.673063170884591]
  EOLO is a novel object detection framework that achieves robust and efficient all-day detection by fusing both RGB and event modalities. Our EOLO framework is built based on a lightweight spiking neural network (SNN) to efficiently leverage the asynchronous property of events.
  arXiv  Detail & Related papers  (2023-09-17T15:14:01Z)
• SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition [42.118434116034194]
  We propose to recognize patterns by fusing RGB frames and event streams simultaneously. Due to the scarce of RGB-Event based classification dataset, we also propose a large-scale PokerEvent dataset.
  arXiv  Detail & Related papers  (2023-08-08T16:15:35Z)
• RGB-D Saliency Detection via Cascaded Mutual Information Minimization [122.8879596830581]
  Existing RGB-D saliency detection models do not explicitly encourage RGB and depth to achieve effective multi-modal learning. We introduce a novel multi-stage cascaded learning framework via mutual information minimization to "explicitly" model the multi-modal information between RGB image and depth data.
  arXiv  Detail & Related papers  (2021-09-15T12:31:27Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.