A Recurrent YOLOv8-based framework for Event-Based Object Detection

• URL: http://arxiv.org/abs/2408.05321v1
• Date: Fri, 9 Aug 2024 20:00:16 GMT
• Title: A Recurrent YOLOv8-based framework for Event-Based Object Detection
• Authors: Diego A. Silva, Kamilya Smagulova, Ahmed Elsheikh, Mohammed E. Fouda, Ahmed M. Eltawil,
• Abstract summary: This study introduces ReYOLOv8, an advanced object detection framework that enhances a frame-based detection system withtemporal modeling capabilities. We implement a low-latency, memory-efficient method for encoding event data to boost the system's performance. We also developed a novel data augmentation technique tailored to leverage the unique attributes of event data, thus improving detection accuracy.
• Score: 4.866548300593921
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object detection is crucial in various cutting-edge applications, such as autonomous vehicles and advanced robotics systems, primarily relying on data from conventional frame-based RGB sensors. However, these sensors often struggle with issues like motion blur and poor performance in challenging lighting conditions. In response to these challenges, event-based cameras have emerged as an innovative paradigm. These cameras, mimicking the human eye, demonstrate superior performance in environments with fast motion and extreme lighting conditions while consuming less power. This study introduces ReYOLOv8, an advanced object detection framework that enhances a leading frame-based detection system with spatiotemporal modeling capabilities. We implemented a low-latency, memory-efficient method for encoding event data to boost the system's performance. We also developed a novel data augmentation technique tailored to leverage the unique attributes of event data, thus improving detection accuracy. Our models outperformed all comparable approaches in the GEN1 dataset, focusing on automotive applications, achieving mean Average Precision (mAP) improvements of 5%, 2.8%, and 2.5% across nano, small, and medium scales, respectively.These enhancements were achieved while reducing the number of trainable parameters by an average of 4.43% and maintaining real-time processing speeds between 9.2ms and 15.5ms. On the PEDRo dataset, which targets robotics applications, our models showed mAP improvements ranging from 9% to 18%, with 14.5x and 3.8x smaller models and an average speed enhancement of 1.67x.

Related papers

• LAM-YOLO: Drones-based Small Object Detection on Lighting-Occlusion Attention Mechanism YOLO [0.9062164411594178]
  LAM-YOLO is an object detection model specifically designed for drone-based images. We introduce a light-occlusion attention mechanism to enhance the visibility of small targets under different lighting conditions. Second, we utilize an improved SIB-IoU as the regression loss function to accelerate model convergence and enhance localization accuracy.
  arXiv  Detail & Related papers  (2024-11-01T10:00:48Z)
• SOD-YOLOv8 -- Enhancing YOLOv8 for Small Object Detection in Traffic Scenes [1.3812010983144802]
  Small Object Detection YOLOv8 (SOD-YOLOv8) is designed for scenarios involving numerous small objects. SOD-YOLOv8 significantly improves small object detection, surpassing widely used models in various metrics. In dynamic real-world traffic scenes, SOD-YOLOv8 demonstrated notable improvements in diverse conditions.
  arXiv  Detail & Related papers  (2024-08-08T23:05:25Z)
• Constellation Dataset: Benchmarking High-Altitude Object Detection for an Urban Intersection [7.419274609612015]
  We introduce Constellation, a dataset of 13K images suitable for research on detection of objects in dense urban streetscapes observed from high-elevation cameras. The dataset addresses the need for curated data to explore problems in small object detection exemplified by the limited pixel footprint of pedestrians. We evaluate contemporary object detection architectures on the dataset, observing that state-of-the-art methods have lower performance in detecting small pedestrians compared to vehicles.
  arXiv  Detail & Related papers  (2024-04-25T18:00:24Z)
• Simple In-place Data Augmentation for Surveillance Object Detection [2.3841361713768077]
  We propose a straightforward augmentation technique tailored for object detection datasets. Our approach focuses on placing objects in the same positions as the originals to ensure its effectiveness.
  arXiv  Detail & Related papers  (2024-04-17T10:20:16Z)
• SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection [53.19618419772467]
  Single-frame infrared small target (SIRST) detection aims to recognize small targets from clutter backgrounds. With the development of Transformer, the scale of SIRST models is constantly increasing. With a rich diversity of infrared small target data, our algorithm significantly improves the model performance and convergence speed.
  arXiv  Detail & Related papers  (2024-03-08T16:14:54Z)
• ShapeAug: Occlusion Augmentation for Event Camera Data [13.634866461329224]
  We present a novel event data augmentation approach for Dynamic Vision Sensors (DVSs) We introduce synthetic events for randomly moving objects in a scene. We test our method on multiple DVS classification datasets, resulting in an improvement of up to 6.5 % in top1-accuracy.
  arXiv  Detail & Related papers  (2024-01-04T13:49:45Z)
• Deformable Neural Radiance Fields using RGB and Event Cameras [65.40527279809474]
  We develop a novel method to model the deformable neural radiance fields using RGB and event cameras. The proposed method uses the asynchronous stream of events and sparse RGB frames. Experiments conducted on both realistically rendered graphics and real-world datasets demonstrate a significant benefit of the proposed method.
  arXiv  Detail & Related papers  (2023-09-15T14:19:36Z)
• Data-driven Feature Tracking for Event Cameras [48.04815194265117]
  We introduce the first data-driven feature tracker for event cameras, which leverages low-latency events to track features detected in a grayscale frame. By directly transferring zero-shot from synthetic to real data, our data-driven tracker outperforms existing approaches in relative feature age by up to 120%. This performance gap is further increased to 130% by adapting our tracker to real data with a novel self-supervision strategy.
  arXiv  Detail & Related papers  (2022-11-23T10:20:11Z)
• Inertial Hallucinations -- When Wearable Inertial Devices Start Seeing Things [82.15959827765325]
  We propose a novel approach to multimodal sensor fusion for Ambient Assisted Living (AAL) We address two major shortcomings of standard multimodal approaches, limited area coverage and reduced reliability. Our new framework fuses the concept of modality hallucination with triplet learning to train a model with different modalities to handle missing sensors at inference time.
  arXiv  Detail & Related papers  (2022-07-14T10:04:18Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing [70.16043883381677]
  In autonomous racing, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres. In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions. We introduce an approach of using synthesised adverse condition datasets in autonomous racing (generated using CycleGAN) to improve the performance of four out of five state-of-the-art detectors.
  arXiv  Detail & Related papers  (2022-01-10T10:02:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.