Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection

• URL: http://arxiv.org/abs/2606.24092v1
• Date: Tue, 23 Jun 2026 03:14:27 GMT
• Title: Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection
• Abstract summary: We propose Progressive Pixel-Neighborhood Deformable Cross-Attention for multispectral feature fusion, termed PNAFusion.<n>PNAFusion incorporates local spatial priors into its architectural design to concentrate feature interaction and alignment on the most relevant neighborhoods.<n>Experiments on FLIR, M3FD, and DroneVehicle show that PNAFusion achieves 84.2, 90.5, and 85.5 mAP@0.5, respectively, under the YOLOv5 detector.
• Score: 4.239735466670748
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective cross-modal feature alignment and interaction are central challenges in multispectral object detection. Although global cross-attention provides strong long-range modeling ability, its quadratic complexity with respect to feature size limits deployment on resource-constrained platforms. We therefore propose Progressive Pixel-Neighborhood Deformable Cross-Attention for multispectral feature fusion, termed PNAFusion. The proposed framework is motivated by two observations: weak misalignment between visible and thermal images is usually concentrated around local neighborhoods, and semantic correspondence across modalities often follows non-linear spatial mappings that fixed receptive fields cannot model well. To address these issues, PNAFusion incorporates local spatial priors into its architectural design to concentrate feature interaction and alignment on the most relevant neighborhoods. Specifically, a Pixel-Neighborhood Cross-Attention (PNCA) module is introduced to avoid redundant global feature matching and suppress background noise. Meanwhile, an Adaptive Deformable Alignment (ADA) module captures non-linear spatial correspondences through learned pixel-wise offsets. These components are further integrated through an iterative feedback mechanism to progressively refine cross-modal feature alignment. Experiments on FLIR, M3FD, and DroneVehicle show that PNAFusion achieves 84.2, 90.5, and 85.5 mAP@0.5, respectively, under the YOLOv5 detector, and further reaches 86.8 mAP@0.5 on FLIR and 90.8 mAP@0.5 on M3FD when transferred to Co-DETR. Efficiency analysis indicates that PNAFusion reduces allocated GPU memory by 33.0\% compared with ICAFusion and reduces theoretical FLOPs from 194.8 G to 156.4 G, although the deformable sampling and iterative refinement introduce additional latency. Our code will be available at https://github.com/DanielQiuTian/PNAFusion.

Related papers

• ESAFusion: LiDAR--4-D Radar Fusion via Local Geometric Complementation and Multiscale Adaptive Interaction for 3-D Object Detection [11.802078221832888]
  LiDAR--4-D radar fusion combines accurate spatial geometry with motion and reflectivity cues from radar, offering a promising solution for 3-D object detection in complex driving environments.<n>We propose ESAFusion, an evidence-aware and scale-adaptive framework that combines local geometric complementation with multiscale adaptive interaction.<n>Experiments on the View-of-Delft (VoD) dataset show that ESAFusion achieves the highest mean average precision (mAP) among the compared methods.
  arXiv  Detail & Related papers  (2026-09-13T15:48:23Z)
• ProtoHGF-Net: Prototype HyperGraph Fusion with Intra-modal Calibration for RGBT Object Detection [18.496442552804123]
  We propose a novel framework that redefines cross-modal fusion as prototype-level semantic interaction.<n>ProtoHGF-Net achieves state-of-the-art performance with 85.9% $mAP_50$, 88.2% $mAP_50$, and 79.1% $mAP_50$, respectively.
  arXiv  Detail & Related papers  (2026-08-12T03:08:19Z)
• Surflo: Consistent 3D Surface Flow Model with Global State [57.57234680235885]
  Surflo decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching.<n>To suppress the local inconsistencies inherent to independent per-point decoding, an inference-time guidance term correlates nearby points by injecting a photometric gradient.<n>Surflo matches or surpasses feed-forward baselines on surface metrics, runs an order of magnitude faster than optimization-based methods that require hundreds of views, and is the only feed-forward approach to combine a global latent with arbitrary-resolution decoding.
  arXiv  Detail & Related papers  (2026-06-11T17:48:38Z)
• RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow [51.43025173196566]
  Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
  arXiv  Detail & Related papers  (2026-04-21T11:32:49Z)
• Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Prediction [9.96771578356719]
  Existing approaches rely on increasingly complex network architectures to regress depth maps.<n>We propose a multilevel conditional random field (CRF) model built upon the Swin Transformer backbone.<n>Our method achieves state-of-the-art performance, reducing Abs Rel to 0.088 ($-$7.4%) and RMSE to 0.316 ($-$5.4%) on NYU Depth v2, while attaining near-perfect threshold accuracy.
  arXiv  Detail & Related papers  (2026-04-03T07:59:26Z)
• LAF-YOLOv10 with Partial Convolution Backbone, Attention-Guided Feature Pyramid, Auxiliary P2 Head, and Wise-IoU Loss for Small Object Detection in Drone Aerial Imagery [0.0]
  Unmanned aerial vehicles serve as primary sensing platforms for surveillance, traffic monitoring, and disaster response.<n>Current detectors struggle with UAV-specific challenges: targets spanning only a few pixels, cluttered backgrounds, heavy occlusion, and strict onboard computational budgets.<n>This study introduces LAF-YOLOv10, built on YOLOv10n, integrating four complementary techniques to improve small-object detection in drone imagery.
  arXiv  Detail & Related papers  (2026-02-13T18:23:54Z)
• Fluxamba: Topology-Aware Anisotropic State Space Models for Geological Lineament Segmentation in Multi-Source Remote Sensing [6.815807403335458]
  We propose a lightweight architecture that introduces a topology-aware feature rectification framework.<n>F Fluxamba achieves a real-time inference speed of over 24 FPS with only 3.4M parameters and 6.3G FLOPs.
  arXiv  Detail & Related papers  (2026-01-24T03:55:21Z)
• UAGLNet: Uncertainty-Aggregated Global-Local Fusion Network with Cooperative CNN-Transformer for Building Extraction [83.48950950780554]
  Building extraction from remote sensing images is a challenging task due to the complex structure variations of buildings.<n>Existing methods employ convolutional or self-attention blocks to capture the multi-scale features in the segmentation models.<n>We present an Uncertainty-Aggregated Global-Local Fusion Network (UAGLNet) to exploit high-quality global-local visual semantics.
  arXiv  Detail & Related papers  (2025-12-15T02:59:16Z)
• DFIR-DETR: Frequency Domain Enhancement and Dynamic Feature Aggregation for Cross-Scene Small Object Detection [16.16000521213211]
  Small object detection in UAV remote sensing images is difficult.<n>Current transformer-based detectors struggle with three critical issues.<n>We introduce DFIR-DETR to tackle these problems through dynamic feature aggregation combined with frequency-domain processing.
  arXiv  Detail & Related papers  (2025-12-08T01:25:10Z)
• An Efficient Aerial Image Detection with Variable Receptive Fields [0.0]
  We propose a transformer-based detector incorporating three key components.<n>VRF-DETR achieves 51.4% mAPtextsubscript50 and 31.8% mAPtextsubscript50:95 with only 13.5M parameters.
  arXiv  Detail & Related papers  (2025-04-21T15:16:13Z)
• PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection [59.355022416218624]
  integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. We propose a novel two-stage 3D object detector, called Point-Voxel Attention Fusion Network (PVAFN) PVAFN uses a multi-pooling strategy to integrate both multi-scale and region-specific information effectively.
  arXiv  Detail & Related papers  (2024-08-26T19:43:01Z)
• DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection [83.18142309597984]
  Lidars and cameras are critical sensors that provide complementary information for 3D detection in autonomous driving. We develop a family of generic multi-modal 3D detection models named DeepFusion, which is more accurate than previous methods.
  arXiv  Detail & Related papers  (2022-03-15T18:46:06Z)
• MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video [75.23812405203778]
  Recent solutions have been introduced to estimate 3D human pose from 2D keypoint sequence by considering body joints among all frames globally to learn-temporal correlation. We propose Mix Mix, which has temporal transformer block to separately model the temporal motion of each joint and a transformer block inter-joint spatial correlation. In addition, the network output is extended from the central frame to entire frames of input video, improving the coherence between the input and output benchmarks.
  arXiv  Detail & Related papers  (2022-03-02T04:20:59Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.