Handwashing Action Detection System for an Autonomous Social Robot

• URL: http://arxiv.org/abs/2210.15804v1
• Date: Thu, 27 Oct 2022 23:46:56 GMT
• Title: Handwashing Action Detection System for an Autonomous Social Robot
• Authors: Sreejith Sasidharan, Pranav Prabha, Devasena Pasupuleti, Anand M Das, Chaitanya Kapoor, Gayathri Manikutty, Praveen Pankajakshan, Bhavani Rao
• Abstract summary: Young children are at an increased risk of contracting contagious diseases such as COVID-19 due to improper hand hygiene. An autonomous social agent that observes children while handwashing and encourages good hand washing practices could provide an opportunity for handwashing behavior to become a habit. We present a human action recognition system, which is part of the vision system of a social robot platform, to assist children in developing a correct handwashing technique.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Young children are at an increased risk of contracting contagious diseases such as COVID-19 due to improper hand hygiene. An autonomous social agent that observes children while handwashing and encourages good hand washing practices could provide an opportunity for handwashing behavior to become a habit. In this article, we present a human action recognition system, which is part of the vision system of a social robot platform, to assist children in developing a correct handwashing technique. A modified convolution neural network (CNN) architecture with Channel Spatial Attention Bilinear Pooling (CSAB) frame, with a VGG-16 architecture as the backbone is trained and validated on an augmented dataset. The modified architecture generalizes well with an accuracy of 90% for the WHO-prescribed handwashing steps even in an unseen environment. Our findings indicate that the approach can recognize even subtle hand movements in the video and can be used for gesture detection and classification in social robotics.

Related papers

• OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer [34.13467792368733]
  We introduce OSMO, a wearable tactile glove designed for human-to-robot skill transfer.<n>We demonstrate that a robot policy trained exclusively on human demonstrations is capable of executing a challenging contact-rich manipulation task.
  arXiv  Detail & Related papers  (2025-12-09T18:56:30Z)
• Giving Robots a Hand: Learning Generalizable Manipulation with Eye-in-Hand Human Video Demonstrations [66.47064743686953]
  Eye-in-hand cameras have shown promise in enabling greater sample efficiency and generalization in vision-based robotic manipulation. Videos of humans performing tasks, on the other hand, are much cheaper to collect since they eliminate the need for expertise in robotic teleoperation. In this work, we augment narrow robotic imitation datasets with broad unlabeled human video demonstrations to greatly enhance the generalization of eye-in-hand visuomotor policies.
  arXiv  Detail & Related papers  (2023-07-12T07:04:53Z)
• Gesture2Path: Imitation Learning for Gesture-aware Navigation [54.570943577423094]
  We present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios.
  arXiv  Detail & Related papers  (2022-09-19T23:05:36Z)
• A Novel IoT-based Framework for Non-Invasive Human Hygiene Monitoring using Machine Learning Techniques [1.4260605984981949]
  This paper presents a novel framework for monitoring human hygiene using vibration sensors. The approach is based on a combination of a geophone sensor, a digitizer, and a cost-efficient computer board in a practical enclosure. Applying a Support Vector Machine for binary classification exhibits a promising accuracy of 95% in the classification of different hygiene habits.
  arXiv  Detail & Related papers  (2022-07-07T18:48:48Z)
• DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video [86.49357517864937]
  We propose DexVIP, an approach to learn dexterous robotic grasping from human-object interaction videos. We do this by curating grasp images from human-object interaction videos and imposing a prior over the agent's hand pose. We demonstrate that DexVIP compares favorably to existing approaches that lack a hand pose prior or rely on specialized tele-operation equipment.
  arXiv  Detail & Related papers  (2022-02-01T00:45:57Z)
• A Deep Learning Based Automated Hand Hygiene Training System [0.12313056815753944]
  WHO recommends a guideline for alcohol-based hand rub in eight steps to ensure that all surfaces of hands are entirely clean. Deep Neural Network (DNN) and machine vision have made it possible to accurately evaluate hand rubbing quality. In this paper, an automated deep learning based hand rub assessment system with real-time feedback is presented.
  arXiv  Detail & Related papers  (2021-12-10T17:01:44Z)
• Automatic Unstructured Handwashing Recognition using Smartwatch to Reduce Contact Transmission of Pathogens [2.578242050187029]
  SARSCoV-2 coronavirus is transmitted through respiratory droplets or by contact. Current smartwatches are able to recognize when a subject is washing or rubbing its hands, in order to monitor parameters such as frequency and duration. Our preliminary results show a classification accuracy of about 95% and of about 94% for respectively deep and standard learning techniques.
  arXiv  Detail & Related papers  (2021-07-28T14:52:45Z)
• Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition [54.749127627191655]
  The ability to recognize human partners is an important social skill to build personalized and long-term human-robot interactions. Deep learning networks have achieved state-of-the-art results and demonstrated to be suitable tools to address such a task. One solution is to make robots learn from their first-hand sensory data with self-supervision.
  arXiv  Detail & Related papers  (2021-03-16T13:50:24Z)
• Where is my hand? Deep hand segmentation for visual self-recognition in humanoid robots [129.46920552019247]
  We propose the use of a Convolution Neural Network (CNN) to segment the robot hand from an image in an egocentric view. We fine-tuned the Mask-RCNN network for the specific task of segmenting the hand of the humanoid robot Vizzy.
  arXiv  Detail & Related papers  (2021-02-09T10:34:32Z)
• Automated Quality Assessment of Hand Washing Using Deep Learning [1.8920934738244022]
  We present neural networks for automatically recognizing the different washing movements defined by the WHO. We train the neural network on a part of a large (2000+ videos) real-world labeled dataset with the different washing movements. Using pre-trained neural network models such as MobileNetV2 and Xception for the task, it is possible to achieve >64 % accuracy in recognizing the different washing movements.
  arXiv  Detail & Related papers  (2020-11-23T13:22:53Z)
• A Developmental Neuro-Robotics Approach for Boosting the Recognition of Handwritten Digits [91.3755431537592]
  Recent evidence shows that a simulation of the children's embodied strategies can improve the machine intelligence too. This article explores the application of embodied strategies to convolutional neural network models in the context of developmental neuro-robotics.
  arXiv  Detail & Related papers  (2020-03-23T14:55:00Z)
• Fully Automated Hand Hygiene Monitoring\\in Operating Room using 3D Convolutional Neural Network [0.0]
  Hand hygiene is one of the most significant factors in preventing hospital acquired infections (HAI) Recent progress in understanding with convolutional neural net (CNN) has increased the application of recognition and detection of human actions. We propose a fully automated hand hygiene monitoring tool of the alcohol-based hand rubbing action of anesthesiologists on OR video usingtemporal features with 3D CNN.
  arXiv  Detail & Related papers  (2020-03-20T03:18:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.