TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection
- URL: http://arxiv.org/abs/2305.16580v3
- Date: Wed, 18 Oct 2023 01:45:06 GMT
- Title: TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection
- Authors: Xue Zhang, Xiao-Han Zhang, Jiacheng Ying, Zehua Sheng, Heng Yu,
Chunguang Li, Hui-Liang Shen
- Abstract summary: We propose a novel target-aware fusion strategy for multispectral pedestrian detection, named TFDet.
Our fusion strategy highlights the pedestrian-related features and suppresses unrelated ones, generating more discriminative fused features. TFDet achieves state-of-the-art performance on both KAIST and LLVIP benchmarks, with an efficiency comparable to the previous state-of-the-art counterpart.
- Score: 21.502127701404792
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Pedestrian detection plays a critical role in computer vision as it
contributes to ensuring traffic safety. Existing methods that rely solely on
RGB images suffer from performance degradation under low-light conditions due
to the lack of useful information. To address this issue, recent multispectral
detection approaches have combined thermal images to provide complementary
information and have obtained enhanced performances. Nevertheless, few
approaches focus on the negative effects of false positives caused by noisy
fused feature maps. Different from them, we comprehensively analyze the impacts
of false positives on the detection performance and find that enhancing feature
contrast can significantly reduce these false positives. In this paper, we
propose a novel target-aware fusion strategy for multispectral pedestrian
detection, named TFDet. Our fusion strategy highlights the pedestrian-related
features and suppresses unrelated ones, generating more discriminative fused
features. TFDet achieves state-of-the-art performance on both KAIST and LLVIP
benchmarks, with an efficiency comparable to the previous state-of-the-art
counterpart. Importantly, TFDet performs remarkably well even under low-light
conditions, which is a significant advancement for ensuring road safety. The
code will be made publicly available at https://github.com/XueZ-phd/TFDet.git.
Related papers
- CFMW: Cross-modality Fusion Mamba for Multispectral Object Detection under Adverse Weather Conditions [17.216501433862014]
Cross-modality images that integrate visible-infrared spectra cues can provide richer information for object detection.
Existing visible-infrared object detection methods severely degrade in severe weather conditions.
We introduce visible-infrared object detection under adverse weather conditions.
arXiv Detail & Related papers (2024-04-25T02:54:11Z) - SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised
Learning for Robust Infrared Small Target Detection [53.19618419772467]
Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds.
With the development of Transformer, the scale of SIRST models is constantly increasing.
With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
arXiv Detail & Related papers (2024-03-08T16:14:54Z) - SpirDet: Towards Efficient, Accurate and Lightweight Infrared Small
Target Detector [60.42293239557962]
We propose SpirDet, a novel approach for efficient detection of infrared small targets.
We employ a new dual-branch sparse decoder to restore the feature map.
Extensive experiments show that the proposed SpirDet significantly outperforms state-of-the-art models.
arXiv Detail & Related papers (2024-02-08T05:06:14Z) - Wildfire Smoke Detection with Cross Contrast Patch Embedding [5.965059322800441]
The Transformer-based deep networks have increasingly shown significant advantages over CNNs.
Low-level information such as color, transparency and texture is very important for smoke recognition.
The fuzzy boundary of smoke makes the positive and negative label assignment for instances in a dilemma.
arXiv Detail & Related papers (2023-11-16T06:53:03Z) - PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant
Semantic Segmentation [50.556961575275345]
We propose a perception-aware fusion framework to promote segmentation robustness in adversarial scenes.
We show that our scheme substantially enhances the robustness, with gains of 15.3% mIOU, compared with advanced competitors.
arXiv Detail & Related papers (2023-08-08T01:55:44Z) - ReDFeat: Recoupling Detection and Description for Multimodal Feature
Learning [51.07496081296863]
We recouple independent constraints of detection and description of multimodal feature learning with a mutual weighting strategy.
We propose a detector that possesses a large receptive field and is equipped with learnable non-maximum suppression layers.
We build a benchmark that contains cross visible, infrared, near-infrared and synthetic aperture radar image pairs for evaluating the performance of features in feature matching and image registration tasks.
arXiv Detail & Related papers (2022-05-16T04:24:22Z) - Illumination and Temperature-Aware Multispectral Networks for
Edge-Computing-Enabled Pedestrian Detection [10.454696553567809]
This study proposes a lightweight Illumination and Temperature-aware Multispectral Network (IT-MN) for accurate and efficient pedestrian detection.
The proposed algorithm is evaluated by comparing with the selected state-of-the-art algorithms using a public dataset collected by in-vehicle cameras.
The results show that the proposed algorithm achieves a low miss rate and inference time at 14.19% and 0.03 seconds per image pair on GPU.
arXiv Detail & Related papers (2021-12-09T17:27:23Z) - Real-Time Anomaly Detection in Edge Streams [49.26098240310257]
We propose MIDAS, which focuses on detecting microcluster anomalies, or suddenly arriving groups of suspiciously similar edges.
We further propose MIDAS-F, to solve the problem by which anomalies are incorporated into the algorithm's internal states.
Experiments show that MIDAS-F has significantly higher accuracy than MIDAS.
arXiv Detail & Related papers (2020-09-17T17:59:27Z) - Anchor-free Small-scale Multispectral Pedestrian Detection [88.7497134369344]
We propose a method for effective and efficient multispectral fusion of the two modalities in an adapted single-stage anchor-free base architecture.
We aim at learning pedestrian representations based on object center and scale rather than direct bounding box predictions.
Results show our method's effectiveness in detecting small-scaled pedestrians.
arXiv Detail & Related papers (2020-08-19T13:13:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.