Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices

• URL: http://arxiv.org/abs/2108.11754v1
• Date: Sun, 22 Aug 2021 01:48:53 GMT
• Title: Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices
• Authors: Agnik Banerjee, Peter Washington, Cezmi Mutlu, Aaron Kline, Dennis P. Wall
• Abstract summary: We optimized and profiled various machine learning models designed for inference on edge devices. Our best model, a MobileNet-V2 network pre-trained on ImageNet, achieved 65.11% balanced accuracy and 64.19% F1-score on CAFE. This balanced accuracy is only 1.79% less than the current state of the art for CAFE, which used a model that contains 26.62x more parameters and was unable to run on the Moto G6.
• Score: 1.996835144477268
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Implementing automated emotion recognition on mobile devices could provide an accessible diagnostic and therapeutic tool for those who struggle to recognize emotion, including children with developmental behavioral conditions such as autism. Although recent advances have been made in building more accurate emotion classifiers, existing models are too computationally expensive to be deployed on mobile devices. In this study, we optimized and profiled various machine learning models designed for inference on edge devices and were able to match previous state of the art results for emotion recognition on children. Our best model, a MobileNet-V2 network pre-trained on ImageNet, achieved 65.11% balanced accuracy and 64.19% F1-score on CAFE, while achieving a 45-millisecond inference latency on a Motorola Moto G6 phone. This balanced accuracy is only 1.79% less than the current state of the art for CAFE, which used a model that contains 26.62x more parameters and was unable to run on the Moto G6, even when fully optimized. This work validates that with specialized design and optimization techniques, machine learning models can become lightweight enough for deployment on mobile devices and still achieve high accuracies on difficult image classification tasks.

Related papers

• MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devices [16.489105620313065]
  MobilePortrait is a one-shot neural head avatars method that reduces learning complexity by integrating external knowledge into both the motion modeling and image synthesis. It achieves state-of-the-art performance with less than one-tenth the computational demand. It has been validated to reach speeds of over 100 FPS on mobile devices and support both video and audio-driven inputs.
  arXiv  Detail & Related papers  (2024-07-08T08:12:57Z)
• StairNet: Visual Recognition of Stairs for Human-Robot Locomotion [2.3811618212533663]
  StairNet is an initiative to support the development of new deep learning models for visual sensing and recognition of stairs. We present an overview of the development of our large-scale dataset with over 515,000 manually labeled images. We show that StairNet can be an effective platform to develop and study new visual perception systems for human-robot locomotion.
  arXiv  Detail & Related papers  (2023-10-31T17:30:57Z)
• MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning [114.66037224769005]
  We present a novel MicroISP model designed specifically for edge devices. The proposed solution is capable of processing up to 32MP photos on recent smartphones using the standard mobile ML libraries. The architecture of the model is flexible, allowing to adjust its complexity to devices of different computational power.
  arXiv  Detail & Related papers  (2022-11-08T17:40:50Z)
• Face Detection on Mobile: Five Implementations and Analysis [0.0]
  We adapt 5 algorithms to mobile, including Viola-Jones (Haar cascade), LBP, HOG, MTCNN, BlazeFace. We provide guidance, which algorithms are the best fit for mobile face access control systems and potentially other mobile applications.
  arXiv  Detail & Related papers  (2022-05-11T15:39:21Z)
• Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report [67.86837649834636]
  We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solution. The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 40-60 ms.
  arXiv  Detail & Related papers  (2021-05-17T13:34:15Z)
• Facial Masks and Soft-Biometrics: Leveraging Face Recognition CNNs for Age and Gender Prediction on Mobile Ocular Images [53.913598771836924]
  We address the use of selfie ocular images captured with smartphones to estimate age and gender. We adapt two existing lightweight CNNs proposed in the context of the ImageNet Challenge. Some networks are further pre-trained for face recognition, for which very large training databases are available.
  arXiv  Detail & Related papers  (2021-03-31T01:48:29Z)
• It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation [19.046126301352274]
  On-device machine learning is becoming a reality thanks to the availability of powerful hardware and model compression techniques. In this work, we observe that a much smaller, personalised model can be employed to fit a specific scenario. We introduce PershonEPEE, a framework that attaches early exits on the model and personalises them on-device.
  arXiv  Detail & Related papers  (2021-02-02T09:10:17Z)
• Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study [5.258326585054865]
  Automated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. Most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces. We designed a strategy to gamify the collection and labeling of child emotion-enriched images to boost the performance of automatic child emotion recognition models.
  arXiv  Detail & Related papers  (2020-12-16T00:08:51Z)
• Real-Time Execution of Large-scale Language Models on Mobile [49.32610509282623]
  We find the best model structure of BERT for a given computation size to match specific devices. Our framework can guarantee the identified model to meet both resource and real-time specifications of mobile devices. Specifically, our model is 5.2x faster on CPU and 4.1x faster on GPU with 0.5-2% accuracy loss compared with BERT-base.
  arXiv  Detail & Related papers  (2020-09-15T01:59:17Z)
• SqueezeFacePoseNet: Lightweight Face Verification Across Different Poses for Mobile Platforms [55.84746218227712]
  Face verification technologies can provide reliable and robust user authentication, given the availability of cameras in mobile devices. Deep Convolutional Neural Networks have resulted in many accurate face verification architectures, but their typical size (hundreds of megabytes) makes them infeasible to be incorporated in downloadable mobile applications. We develop a lightweight face recognition network of just a few megabytes that can operate with sufficient accuracy in comparison to much larger models.
  arXiv  Detail & Related papers  (2020-07-16T19:02:38Z)
• A Data and Compute Efficient Design for Limited-Resources Deep Learning [68.55415606184]
  equivariant neural networks have gained increased interest in the deep learning community. They have been successfully applied in the medical domain where symmetries in the data can be effectively exploited to build more accurate and robust models. Mobile, on-device implementations of deep learning solutions have been developed for medical applications. However, equivariant models are commonly implemented using large and computationally expensive architectures, not suitable to run on mobile devices. In this work, we design and test an equivariant version of MobileNetV2 and further optimize it with model quantization to enable more efficient inference.
  arXiv  Detail & Related papers  (2020-04-21T00:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.