Efficient Fourier Filtering Network with Contrastive Learning for UAV-based Unaligned Bi-modal Salient Object Detection

• URL: http://arxiv.org/abs/2411.03728v2
• Date: Thu, 17 Apr 2025 07:11:34 GMT
• Title: Efficient Fourier Filtering Network with Contrastive Learning for UAV-based Unaligned Bi-modal Salient Object Detection
• Abstract summary: Unmanned aerial vehicle (UAV)-based bi-modal salient object detection (BSOD) aims to segment salient objects in a scene utilizing complementary cues in unaligned RGB and thermal image pairs.<n>We propose an efficient Fourier filter network with contrastive learning that achieves both real-time and accurate performance.<n>Our proposed model, AlignSal, reduces the number of parameters by 70.0%, decreases the floating point operations by 49.4%, and increases the inference speed by 152.5%.
• Score: 8.12311871820753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unmanned aerial vehicle (UAV)-based bi-modal salient object detection (BSOD) aims to segment salient objects in a scene utilizing complementary cues in unaligned RGB and thermal image pairs. However, the high computational expense of existing UAV-based BSOD models limits their applicability to real-world UAV devices. To address this problem, we propose an efficient Fourier filter network with contrastive learning that achieves both real-time and accurate performance. Specifically, we first design a semantic contrastive alignment loss to align the two modalities at the semantic level, which facilitates mutual refinement in a parameter-free way. Second, inspired by the fast Fourier transform that obtains global relevance in linear complexity, we propose synchronized alignment fusion, which aligns and fuses bi-modal features in the channel and spatial dimensions by a hierarchical filtering mechanism. Our proposed model, AlignSal, reduces the number of parameters by 70.0%, decreases the floating point operations by 49.4%, and increases the inference speed by 152.5% compared to the cutting-edge BSOD model (i.e., MROS). Extensive experiments on the UAV RGB-T 2400 and seven bi-modal dense prediction datasets demonstrate that AlignSal achieves both real-time inference speed and better performance and generalizability compared to nineteen state-of-the-art models across most evaluation metrics. In addition, our ablation studies further verify AlignSal's potential in boosting the performance of existing aligned BSOD models on UAV-based unaligned data. The code is available at: https://github.com/JoshuaLPF/AlignSal.

Related papers

• Towards UAV Detection in the Real World: A New Multispectral Dataset UAVNet-MS and a New Method [80.50361258830928]
  UAVNet-MS is the first multispectral dataset for fine-grained small-UAV detection.<n>We propose MFDNet, a dual-stream baseline addressing array-induced parallax and spatial-spectral fusion.<n>This work provides foundational dataset, strong baseline, and benchmark for multispectral UAV monitoring research.
  arXiv  Detail & Related papers  (2026-05-20T09:49:04Z)
• UAV-DETR: DETR for Anti-Drone Target Detection [18.80742672365221]
  We introduce UAV-DETR, a novel framework that integrates a small-target-friendly architecture with real-time detection capabilities.<n>Specifically, UAV-DETR features a WTConv-enhanced backbone and a Sliding Window Self-Attention (SWSA-IFI) encoder, capturing the high-frequency structural details of tiny targets.<n>To further enhance accuracy, UAV-DETR incorporates a hybrid Inner-CIoU and NWD loss strategy, mitigating the extreme sensitivity of standard IoU metrics to minor positional deviations.
  arXiv  Detail & Related papers  (2026-03-24T06:29:51Z)
• LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data [0.0]
  This paper addresses the limitations of current vision-based rail defect detection methods.<n>We propose a Lightweight Pyramid Cross-Attention Network (LPCANet) that leverages RGB-D data for efficient and accurate defect identification.<n>LPCANet achieves state-of-the-art performance with only 9.90 million parameters, 2.50 G FLOPs, and 162.60 fps inference speed.
  arXiv  Detail & Related papers  (2026-01-14T03:35:09Z)
• Light-Weight Cross-Modal Enhancement Method with Benchmark Construction for UAV-based Open-Vocabulary Object Detection [6.443926939309045]
  We propose a complete UAV-oriented solution that combines both dataset construction and model innovation.<n>First, we design a refined UAV-Label Engine, which efficiently resolves annotation redundancy, inconsistency, and ambiguity.<n>Second, we introduce the Cross-Attention Gated Enhancement (CAGE) module, a lightweight dual-path fusion design that integrates cross-attention, adaptive gating, and global FiLM modulation for robust textvision alignment.
  arXiv  Detail & Related papers  (2025-09-07T10:59:02Z)
• UAV Individual Identification via Distilled RF Fingerprints-Based LLM in ISAC Networks [60.16924915676577]
  Unmanned aerial vehicle (UAV) individual (ID) identification is a critical security surveillance strategy in low-altitude integrated sensing and communication (ISAC) networks.<n>We propose a novel dynamic knowledge distillation (KD)-enabled wireless radio frequency fingerprint large language model (RFF-LLM) framework for UAV ID identification.<n>Experiment results show that the proposed framework achieves 98.38% ID identification accuracy with merely 0.15 million parameters and 2.74 ms response time.
  arXiv  Detail & Related papers  (2025-08-18T03:14:44Z)
• Wireless Federated Learning over UAV-enabled Integrated Sensing and Communication [2.8203310972866382]
  This paper studies a new latency optimization problem in unmanned aerial vehicles (UAVs)-enabled federated learning (FL) with integrated sensing and communication. We develop a simple yet efficient iterative algorithm to find a high-quality approximate solution, saving system latency up to 68.54% compared to benchmark schemes.
  arXiv  Detail & Related papers  (2024-11-01T14:25:24Z)
• DiRecNetV2: A Transformer-Enhanced Network for Aerial Disaster Recognition [4.678150356894011]
  integration of Unmanned Aerial Vehicles with artificial intelligence (AI) models for aerial imagery processing in disaster assessment requires exceptional accuracy, computational efficiency, and real-time processing capabilities. Traditionally Convolutional Neural Networks (CNNs) demonstrate efficiency in local feature extraction but are limited by their potential for global context interpretation. Vision Transformers (ViTs) show promise for improved global context interpretation through the use of attention mechanisms, although they still remain underinvestigated in UAV-based disaster response applications.
  arXiv  Detail & Related papers  (2024-10-17T15:25:13Z)
• SOOD++: Leveraging Unlabeled Data to Boost Oriented Object Detection [59.868772767818975]
  We propose a simple yet effective Semi-supervised Oriented Object Detection method termed SOOD++. Specifically, we observe that objects from aerial images are usually arbitrary orientations, small scales, and aggregation. Extensive experiments conducted on various multi-oriented object datasets under various labeled settings demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-07-01T07:03:51Z)
• SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection [53.19618419772467]
  Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds. With the development of Transformer, the scale of SIRST models is constantly increasing. With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
  arXiv  Detail & Related papers  (2024-03-08T16:14:54Z)
• Revisiting Stereo Triangulation in UAV Distance Estimation [5.656973345209692]
  We build and present a UAVDE dataset for UAV distance estimation, in which distance between two UAVs is obtained by UWB sensors. We propose a novel position correction module, which can directly predict the offset between the observed positions and the actual ones. We conduct extensive experiments on UAVDE, and our method can achieve a significant performance improvement over a strong baseline.
  arXiv  Detail & Related papers  (2023-06-15T08:22:05Z)
• Drone Referring Localization: An Efficient Heterogeneous Spatial Feature Interaction Method For UAV Self-Localization [22.94589565476653]
  We propose an efficient heterogeneous spatial feature interaction method, termed Drone Referring localization (DRL) Unlike conventional methods that treat different data sources in isolation, DRL facilitates the learnable interaction of heterogeneous features. Compared to traditional IR methods, DRL achieves superior localization accuracy (MA@20 +9.4%) while significantly reducing computational time (1/7) and storage overhead (2/3)
  arXiv  Detail & Related papers  (2022-08-13T03:25:50Z)
• Anchor Retouching via Model Interaction for Robust Object Detection in Aerial Images [15.404024559652534]
  We present an effective Dynamic Enhancement Anchor (DEA) network to construct a novel training sample generator. Our method achieves state-of-the-art performance in accuracy with moderate inference speed and computational overhead for training.
  arXiv  Detail & Related papers  (2021-12-13T14:37:20Z)
• Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM [0.0]
  We propose an efficient two-stream deep learning architecture leveraging Separable Convolutional LSTM (SepConvLSTM) and pre-trained MobileNet. SepConvLSTM is constructed by replacing convolution operation at each gate of ConvLSTM with a depthwise separable convolution. Our model outperforms the accuracy on the larger and more challenging RWF-2000 dataset by more than a 2% margin.
  arXiv  Detail & Related papers  (2021-02-21T12:01:48Z)
• DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection [104.50425501764806]
  We introduce a large-scale dataset to enable versatile applications for light field saliency detection. We present an asymmetrical two-stream model consisting of the Focal stream and RGB stream. Experiments demonstrate that our Focal stream achieves state-of-the-arts performance.
  arXiv  Detail & Related papers  (2020-12-30T11:53:27Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)
• Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection [91.43066633305662]
  The main purpose of RGB-D salient object detection (SOD) is how to better integrate and utilize cross-modal fusion information. In this paper, we explore these issues from a new perspective. We implement a kind of more flexible and efficient multi-scale cross-modal feature processing.
  arXiv  Detail & Related papers  (2020-07-13T07:59:55Z)
• Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms [98.78553146823829]
  Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks. In this paper, a novel framework is proposed to implement distributed learning (FL) algorithms within a UAV swarm.
  arXiv  Detail & Related papers  (2020-02-19T14:04:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.