The Use of AI for Thermal Emotion Recognition: A Review of Problems and Limitations in Standard Design and Data

• URL: http://arxiv.org/abs/2009.10589v1
• Date: Tue, 22 Sep 2020 14:58:59 GMT
• Title: The Use of AI for Thermal Emotion Recognition: A Review of Problems and Limitations in Standard Design and Data
• Authors: Catherine Ordun, Edward Raff, Sanjay Purushotham
• Abstract summary: With the increased attention on thermal imagery for Covid-19 screening, the public sector may believe there are new opportunities to exploit thermal as a modality for computer vision and AI. This paper takes the reader on a short review of machine learning in thermal FER and the limitations of collecting and developing thermal FER data for AI training.
• Score: 36.33347149799959
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the increased attention on thermal imagery for Covid-19 screening, the public sector may believe there are new opportunities to exploit thermal as a modality for computer vision and AI. Thermal physiology research has been ongoing since the late nineties. This research lies at the intersections of medicine, psychology, machine learning, optics, and affective computing. We will review the known factors of thermal vs. RGB imaging for facial emotion recognition. But we also propose that thermal imagery may provide a semi-anonymous modality for computer vision, over RGB, which has been plagued by misuse in facial recognition. However, the transition to adopting thermal imagery as a source for any human-centered AI task is not easy and relies on the availability of high fidelity data sources across multiple demographics and thorough validation. This paper takes the reader on a short review of machine learning in thermal FER and the limitations of collecting and developing thermal FER data for AI training. Our motivation is to provide an introductory overview into recent advances for thermal FER and stimulate conversation about the limitations in current datasets.

Related papers

• MISFIT-V: Misaligned Image Synthesis and Fusion using Information from Thermal and Visual [2.812395851874055]
  This work presents Misaligned Image Synthesis and Fusion using Information from Thermal and Visual (MISFIT-V) It is a novel two-pronged unsupervised deep learning approach that utilizes a Generative Adversarial Network (GAN) and a cross-attention mechanism to capture the most relevant features from each modality. Experimental results show MISFIT-V offers enhanced robustness against misalignment and poor lighting/thermal environmental conditions.
  arXiv  Detail & Related papers  (2023-09-22T23:41:24Z)
• A Generative Approach for Image Registration of Visible-Thermal (VT) Cancer Faces [77.77475333490744]
  We modernize the classic computer vision task of image registration by applying and modifying a generative alignment algorithm. We demonstrate that the quality of thermal images produced in the generative AI downstream task of Visible-to-Thermal (V2T) image translation significantly improves up to 52.5%.
  arXiv  Detail & Related papers  (2023-08-23T17:39:58Z)
• What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging [22.923237551192834]
  We collect the first RGB-Thermal dataset for human motion analysis, dubbed Thermal-IM. We develop a three-stage neural network model for accurate past human pose estimation.
  arXiv  Detail & Related papers  (2023-04-26T16:23:10Z)
• A Novel Fully Annotated Thermal Infrared Face Dataset: Recorded in Various Environment Conditions and Distances From The Camera [3.2872586139884623]
  This article presents a novel public dataset on facial thermography, which we call it Charlotte-ThermalFace. Charlotte-ThermalFace contains more than10000 infrared thermal images in varying thermal conditions, several distances from the camera, and different head positions. The data is fully annotated with the facial landmarks, ambient temperature, relative humidity, the air speed of the room, distance to the camera, and subject thermal sensation at the time of capturing each image.
  arXiv  Detail & Related papers  (2022-04-29T17:57:54Z)
• Machine Learning-Based Automated Thermal Comfort Prediction: Integration of Low-Cost Thermal and Visual Cameras for Higher Accuracy [3.2872586139884623]
  Real-time feedback system is needed to provide data about occupants' comfort conditions. New solutions are required to bring a more holistic view toward non-intrusive thermal scanning.
  arXiv  Detail & Related papers  (2022-04-14T15:30:16Z)
• Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion [78.19156040783061]
  Self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. The inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. We propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency.
  arXiv  Detail & Related papers  (2022-01-12T09:49:24Z)
• Generating Thermal Human Faces for Physiological Assessment Using Thermal Sensor Auxiliary Labels [21.920079976038163]
  Thermal images reveal medically important physiological information about human stress, signs of inflammation, and emotional mood that cannot be seen on visible images. We introduce favtGAN, a VT GAN which uses the pix2pix image translation model with an auxiliary sensor label prediction network for generating thermal faces from visible images. Experiments on these combined datasets show that favtGAN demonstrates an increase in SSIM and PSNR scores of generated thermal faces, compared to training on a single face dataset alone.
  arXiv  Detail & Related papers  (2021-06-15T12:32:52Z)
• A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset [62.193924313292875]
  We present the DEVCOM Army Research Laboratory Visible-Thermal Face dataset (ARL-VTF) With over 500,000 images from 395 subjects, the ARL-VTF dataset represents to the best of our knowledge, the largest collection of paired visible and thermal face images to date. This paper presents benchmark results and analysis on thermal face landmark detection and thermal-to-visible face verification by evaluating state-of-the-art models on the ARL-VTF dataset.
  arXiv  Detail & Related papers  (2021-01-07T17:17:12Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.