Drone Object Detection Using RGB/IR Fusion

• URL: http://arxiv.org/abs/2201.03786v1
• Date: Tue, 11 Jan 2022 05:15:59 GMT
• Title: Drone Object Detection Using RGB/IR Fusion
• Authors: Lizhi Yang, Ruhang Ma, Avideh Zakhor
• Abstract summary: We develop strategies for creating synthetic IR images using the AIRSim simulation engine and CycleGAN. We utilize an illumination-aware fusion framework to fuse RGB and IR images for object detection on the ground. Our solution is implemented on an NVIDIA Jetson Xavier running on an actual drone, requiring about 28 milliseconds of processing per RGB/IR image pair.
• Score: 1.5469452301122175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object detection using aerial drone imagery has received a great deal of attention in recent years. While visible light images are adequate for detecting objects in most scenarios, thermal cameras can extend the capabilities of object detection to night-time or occluded objects. As such, RGB and Infrared (IR) fusion methods for object detection are useful and important. One of the biggest challenges in applying deep learning methods to RGB/IR object detection is the lack of available training data for drone IR imagery, especially at night. In this paper, we develop several strategies for creating synthetic IR images using the AIRSim simulation engine and CycleGAN. Furthermore, we utilize an illumination-aware fusion framework to fuse RGB and IR images for object detection on the ground. We characterize and test our methods for both simulated and actual data. Our solution is implemented on an NVIDIA Jetson Xavier running on an actual drone, requiring about 28 milliseconds of processing per RGB/IR image pair.

Related papers

• IRSAM: Advancing Segment Anything Model for Infrared Small Target Detection [55.554484379021524]
  Infrared Small Target Detection (IRSTD) task falls short in achieving satisfying performance due to a notable domain gap between natural and infrared images. We propose the IRSAM model for IRSTD, which improves SAM's encoder-decoder architecture to learn better feature representation of infrared small objects.
  arXiv  Detail & Related papers  (2024-07-10T10:17:57Z)
• The Solution for the GAIIC2024 RGB-TIR object detection Challenge [5.625794757504552]
  RGB-TIR object detection aims to utilize both RGB and TIR images for complementary information during detection. Our proposed method achieved an mAP score of 0.516 and 0.543 on A and B benchmarks respectively.
  arXiv  Detail & Related papers  (2024-07-04T12:08:36Z)
• Real Time Human Detection by Unmanned Aerial Vehicles [0.0]
  Two crucial data sources for public security are the thermal infrared (TIR) remote sensing photos and videos produced by unmanned aerial vehicles (UAVs) due to the small scale of the target, complex scene information, low resolution relative to the viewable videos, and dearth of publicly available labeled datasets and training models, their object detection procedure is still difficult. A UAV TIR object detection framework for pictures and videos is suggested in this study.
  arXiv  Detail & Related papers  (2024-01-06T18:28:01Z)
• Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection [10.460296317901662]
  We find detection in aerial RGB-IR images suffers from cross-modal weakly misalignment problems. We propose a Translation-Scale-Rotation Alignment (TSRA) module to address the problem. A two-stream feature alignment detector (TSFADet) based on the TSRA module is constructed for RGB-IR object detection in aerial images.
  arXiv  Detail & Related papers  (2022-09-28T03:06:18Z)
• Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images [89.81919625224103]
  Training deep models for RGB-D salient object detection (SOD) often requires a large number of labeled RGB-D images. We present a Dual-Semi RGB-D Salient Object Detection Network (DS-Net) to leverage unlabeled RGB images for boosting RGB-D saliency detection.
  arXiv  Detail & Related papers  (2022-01-01T03:02:27Z)
• Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds [155.388487263872]
  We propose a new infrared small-dim target detection method with the transformer. We adopt the self-attention mechanism of the transformer to learn the interaction information of image features in a larger range. We also design a feature enhancement module to learn more features of small-dim targets.
  arXiv  Detail & Related papers  (2021-09-29T12:23:41Z)
• EagerMOT: 3D Multi-Object Tracking via Sensor Fusion [68.8204255655161]
  Multi-object tracking (MOT) enables mobile robots to perform well-informed motion planning and navigation by localizing surrounding objects in 3D space and time. Existing methods rely on depth sensors (e.g., LiDAR) to detect and track targets in 3D space, but only up to a limited sensing range due to the sparsity of the signal. We propose EagerMOT, a simple tracking formulation that integrates all available object observations from both sensor modalities to obtain a well-informed interpretation of the scene dynamics.
  arXiv  Detail & Related papers  (2021-04-29T22:30:29Z)
• Multispectral Object Detection with Deep Learning [7.592218846348004]
  In this work, we have taken images with both the Thermal and NIR spectrum for the object detection task. We train the YOLO v3 network from scratch to detect an object from multi-spectral images.
  arXiv  Detail & Related papers  (2021-02-05T11:39:14Z)
• Perceiving Traffic from Aerial Images [86.994032967469]
  We propose an object detection method called Butterfly Detector that is tailored to detect objects in aerial images. We evaluate our Butterfly Detector on two publicly available UAV datasets (UAVDT and VisDrone 2019) and show that it outperforms previous state-of-the-art methods while remaining real-time.
  arXiv  Detail & Related papers  (2020-09-16T11:37:43Z)
• Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning [59.19469551774703]
  Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. We construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night.
  arXiv  Detail & Related papers  (2020-03-05T05:29:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.