Multiple Object Detection and Tracking in Panoramic Videos for Cycling Safety Analysis

• URL: http://arxiv.org/abs/2407.15199v1
• Date: Sun, 21 Jul 2024 15:37:55 GMT
• Title: Multiple Object Detection and Tracking in Panoramic Videos for Cycling Safety Analysis
• Authors: Jingwei Guo, Meihui Wang, Ilya Ilyankou, Natchapon Jongwiriyanurak, Xiaowei Gao, Nicola Christie, James Haworth,
• Abstract summary: The project proposed and implemented a three-step methodology to improve the prediction performance of the pre-trained object detection models. The proposed methodology improves the average precision of YOLO v5m6 and Faster RCNN-FPN under any input resolution setting. When detecting the overtakes in the test videos, it achieves the F-score of 0.88.
• Score: 0.06282171844772422
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Panoramic cycling videos can record 360{\deg} views around the cyclists. Thus, it is essential to conduct automatic road user analysis on them using computer vision models to provide data for studies on cycling safety. However, the features of panoramic data such as severe distortions, large number of small objects and boundary continuity have brought great challenges to the existing CV models, including poor performance and evaluation methods that are no longer applicable. In addition, due to the lack of data with annotations, it is not easy to re-train the models. In response to these problems, the project proposed and implemented a three-step methodology: (1) improve the prediction performance of the pre-trained object detection models on panoramic data by projecting the original image into 4 perspective sub-images; (2) introduce supports for boundary continuity and category information into DeepSORT, a commonly used multiple object tracking model, and set an improved detection model as its detector; (3) using the tracking results, develop an application for detecting the overtaking behaviour of the surrounding vehicles. Evaluated on the panoramic cycling dataset built by the project, the proposed methodology improves the average precision of YOLO v5m6 and Faster RCNN-FPN under any input resolution setting. In addition, it raises MOTA and IDF1 of DeepSORT by 7.6\% and 9.7\% respectively. When detecting the overtakes in the test videos, it achieves the F-score of 0.88. The code is available on GitHub at github.com/cuppp1998/360_object_tracking to ensure the reproducibility and further improvements of results.

Related papers

• Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection [11.33083039877258]
  3D semantic occupancy prediction aims to forecast detailed geometric and semantic information of the surrounding environment for autonomous vehicles. We introduce an additional 3D supervision signal by incorporating an additional 3D object detection auxiliary branch. Our approach attains state-of-the-art results, achieving an IoU score of 31.73% and a mIoU score of 20.91%.
  arXiv  Detail & Related papers  (2025-04-07T05:08:22Z)
• Tracking Meets Large Multimodal Models for Driving Scenario Understanding [76.71815464110153]
  Large Multimodal Models (LMMs) have recently gained prominence in autonomous driving research. We propose to integrate tracking information as an additional input to recover 3D spatial and temporal details. We introduce a novel approach for embedding this tracking information into LMMs to enhance their understanding of driving scenarios.
  arXiv  Detail & Related papers  (2025-03-18T17:59:12Z)
• Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning [13.613407983544427]
  We introduce a robust model designed to withstand changes in camera position within the vehicle. Our Driver Behavior Monitoring Network (DBMNet) relies on a lightweight backbone and integrates a disentanglement module. Experiments conducted on the daytime and nighttime subsets of the 100-Driver dataset validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-11-20T10:27:12Z)
• Optimizing YOLO Architectures for Optimal Road Damage Detection and Classification: A Comparative Study from YOLOv7 to YOLOv10 [0.0]
  This paper presents a comprehensive workflow for road damage detection using deep learning models. To accommodate hardware limitations, large images are cropped, and lightweight models are utilized. The proposed approach employs multiple model architectures, including a custom YOLOv7 model with Coordinate Attention layers and a Tiny YOLOv7 model.
  arXiv  Detail & Related papers  (2024-10-10T22:55:12Z)
• Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection [9.708971995966476]
  This paper introduces a two-stage training strategy to address these challenges. Our approach initially trains a model on the large-scale synthetic dataset, RoadSense3D. We fine-tune the model on a combination of real-world datasets to enhance its adaptability to practical conditions.
  arXiv  Detail & Related papers  (2024-08-28T08:44:58Z)
• TAPVid-3D: A Benchmark for Tracking Any Point in 3D [63.060421798990845]
  We introduce a new benchmark, TAPVid-3D, for evaluating the task of Tracking Any Point in 3D. This benchmark will serve as a guidepost to improve our ability to understand precise 3D motion and surface deformation from monocular video.
  arXiv  Detail & Related papers  (2024-07-08T13:28:47Z)
• V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection [73.37781484123536]
  We introduce a highly performant 3D object detector for point clouds using the DETR framework. To address the limitation, we introduce a novel 3D Relative Position (3DV-RPE) method. We show exceptional results on the challenging ScanNetV2 benchmark.
  arXiv  Detail & Related papers  (2023-08-08T17:14:14Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• Multi-Camera Multiple 3D Object Tracking on the Move for Autonomous Vehicles [17.12321292167318]
  It is important for object detection and tracking to address new challenges, such as achieving consistent results across views of cameras. This work presents a new Global Association Graph Model with Link Prediction approach to predict existing tracklets location and link detections with tracklets. Our model exploits to improve the detection accuracy of a standard 3D object detector in the nuScenes detection challenge.
  arXiv  Detail & Related papers  (2022-04-19T22:50:36Z)
• Weakly Supervised Training of Monocular 3D Object Detectors Using Wide Baseline Multi-view Traffic Camera Data [19.63193201107591]
  7DoF prediction of vehicles at an intersection is an important task for assessing potential conflicts between road users. We develop an approach using a weakly supervised method of fine tuning 3D object detectors for traffic observation cameras. Our method achieves vehicle 7DoF pose prediction accuracy on our dataset comparable to the top performing monocular 3D object detectors on autonomous vehicle datasets.
  arXiv  Detail & Related papers  (2021-10-21T08:26:48Z)
• Learnable Online Graph Representations for 3D Multi-Object Tracking [156.58876381318402]
  We propose a unified and learning based approach to the 3D MOT problem. We employ a Neural Message Passing network for data association that is fully trainable. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
  arXiv  Detail & Related papers  (2021-04-23T17:59:28Z)
• InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information Modeling [65.47126868838836]
  We propose a novel 3D object detection framework with dynamic information modeling. Coarse predictions are generated in the first stage via a voxel-based region proposal network. Experiments are conducted on the large-scale nuScenes 3D detection benchmark.
  arXiv  Detail & Related papers  (2020-07-16T18:27:08Z)
• Road Curb Detection and Localization with Monocular Forward-view Vehicle Camera [74.45649274085447]
  We propose a robust method for estimating road curb 3D parameters using a calibrated monocular camera equipped with a fisheye lens. Our approach is able to estimate the vehicle to curb distance in real time with mean accuracy of more than 90%.
  arXiv  Detail & Related papers  (2020-02-28T00:24:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.