Development, Optimization, and Deployment of Thermal Forward Vision Systems for Advance Vehicular Applications on Edge Devices

• URL: http://arxiv.org/abs/2301.07613v1
• Date: Wed, 18 Jan 2023 15:45:33 GMT
• Title: Development, Optimization, and Deployment of Thermal Forward Vision Systems for Advance Vehicular Applications on Edge Devices
• Authors: Muhammad Ali Farooq, Waseem Shariff, Faisal Khan, Peter Corcoran
• Abstract summary: We have proposed a thermal tiny-YOLO multi-class object detection (TTYMOD) system as a smart forward sensing system using an end-to-end YOLO deep learning framework. The system is trained on large-scale thermal public as well as newly gathered novel open-sourced dataset comprising of more than 35,000 distinct thermal frames. The efficacy of a thermally tuned nano network is quantified using various qualitative metrics which include mean precision, frames per second rate, and average inference time.
• Score: 0.3058685580689604
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this research work, we have proposed a thermal tiny-YOLO multi-class object detection (TTYMOD) system as a smart forward sensing system that should remain effective in all weather and harsh environmental conditions using an end-to-end YOLO deep learning framework. It provides enhanced safety and improved awareness features for driver assistance. The system is trained on large-scale thermal public datasets as well as newly gathered novel open-sourced dataset comprising of more than 35,000 distinct thermal frames. For optimal training and convergence of YOLO-v5 tiny network variant on thermal data, we have employed different optimizers which include stochastic decent gradient (SGD), Adam, and its variant AdamW which has an improved implementation of weight decay. The performance of thermally tuned tiny architecture is further evaluated on the public as well as locally gathered test data in diversified and challenging weather and environmental conditions. The efficacy of a thermally tuned nano network is quantified using various qualitative metrics which include mean average precision, frames per second rate, and average inference time. Experimental outcomes show that the network achieved the best mAP of 56.4% with an average inference time/ frame of 4 milliseconds. The study further incorporates optimization of tiny network variant using the TensorFlow Lite quantization tool this is beneficial for the deployment of deep learning architectures on the edge and mobile devices. For this study, we have used a raspberry pi 4 computing board for evaluating the real-time feasibility performance of an optimized version of the thermal object detection network for the automotive sensor suite. The source code, trained and optimized models and complete validation/ testing results are publicly available at https://github.com/MAli-Farooq/Thermal-YOLO-And-Model-Optimization-Using-TensorFlowLite.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Comparison of edge computing methods in Internet of Things architectures for efficient estimation of indoor environmental parameters with Machine Learning [0.0]
  Two methods are proposed to implement lightweight Machine Learning models that estimate indoor environmental quality (IEQ) parameters. Their implementation is based on centralised and distributed parallel IoT architectures, connected via wireless. The training and testing of ML models is accomplished with experiments focused on small temperature and illuminance datasets.
  arXiv  Detail & Related papers  (2024-02-07T21:15:18Z)
• Small Temperature is All You Need for Differentiable Architecture Search [8.93957397187611]
  Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature.
  arXiv  Detail & Related papers  (2023-06-12T04:01:57Z)
• Online Data Selection for Federated Learning with Limited Storage [53.46789303416799]
  Federated Learning (FL) has been proposed to achieve distributed machine learning among networked devices. The impact of on-device storage on the performance of FL is still not explored. In this work, we take the first step to consider the online data selection for FL with limited on-device storage.
  arXiv  Detail & Related papers  (2022-09-01T03:27:33Z)
• Evaluation of Thermal Imaging on Embedded GPU Platforms for Application in Vehicular Assistance Systems [0.5156484100374058]
  This study is focused on evaluating the real-time performance of thermal object detection for smart and safe vehicular systems. A novel large-scale thermal dataset comprising of > 35,000 distinct frames is acquired. The effectiveness of trained networks is validated on extensive test data using various quantitative metrics.
  arXiv  Detail & Related papers  (2022-01-05T15:36:25Z)
• Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo [105.05232615226602]
  We leverage differentiable neural architecture search (NAS) strategy to find uncalibrated PS architecture automatically. Experiments on the DiLiGenT dataset show that the automatically searched neural architectures performance compares favorably with the state-of-the-art uncalibrated PS methods.
  arXiv  Detail & Related papers  (2021-10-11T21:22:17Z)
• Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learning [64.92447072894055]
  Infrared (IR) cameras are robust under adverse illumination and lighting conditions. We propose an algorithm meta-learning framework to improve existing UDA methods. We produce a state-of-the-art thermal detector for the KAIST and DSIAC datasets.
  arXiv  Detail & Related papers  (2021-10-07T02:28:18Z)
• Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS) [0.5156484100374058]
  Object detection in thermal infrared spectrum provides more reliable data source in low-lighting conditions and different weather conditions. This paper is about exploring and adapting state-of-the-art object and vision framework on thermal vision with seven distinct classes for advanced driver-assistance systems (ADAS) The trained network variants on public datasets are validated on test data with three different test approaches. The efficacy of trained networks is tested on locally gathered novel test-data captured with an uncooled LWIR prototype thermal camera.
  arXiv  Detail & Related papers  (2021-09-20T21:38:55Z)
• Exploiting Adam-like Optimization Algorithms to Improve the Performance of Convolutional Neural Networks [82.61182037130405]
  gradient descent (SGD) is the main approach for training deep networks. In this work, we compare Adam based variants based on the difference between the present and the past gradients. We have tested ensemble of networks and the fusion with ResNet50 trained with gradient descent.
  arXiv  Detail & Related papers  (2021-03-26T18:55:08Z)
• A Sequential Modelling Approach for Indoor Temperature Prediction and Heating Control in Smart Buildings [4.759925918369102]
  This paper proposes a learning-based framework for sequentially applying the data-driven statistical methods to predict indoor temperature. Experiments demonstrate the effectiveness of the modelling approach and control algorithm, and reveal the promising potential of the mixed data-driven approach in smart building applications.
  arXiv  Detail & Related papers  (2020-09-21T13:20:27Z)
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network [55.852401381113786]
  This paper proposes LaMI (Latency Model Inpainting), a novel framework to generate a comprehensive-temporal quality framework for wireless access latency of connected vehicles. LaMI adopts the idea from image inpainting and synthesizing and can reconstruct the missing latency samples by a two-step procedure. In particular, it first discovers the spatial correlation between samples collected in various regions using a patching-based approach and then feeds the original and highly correlated samples into a Varienational Autocoder (VAE)
  arXiv  Detail & Related papers  (2020-03-16T03:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.