Vision-Based Activity Recognition in Children with Autism-Related Behaviors

• URL: http://arxiv.org/abs/2208.04206v1
• Date: Mon, 8 Aug 2022 15:12:27 GMT
• Title: Vision-Based Activity Recognition in Children with Autism-Related Behaviors
• Authors: Pengbo Wei, David Ahmedt-Aristizabal, Harshala Gammulle, Simon Denman, Mohammad Ali Armin
• Abstract summary: We demonstrate the effect of a region-based computer vision system to help clinicians and parents analyze a child's behavior. The data is pre-processed by detecting the target child in the video to reduce the impact of background noise. Motivated by the effectiveness of temporal convolutional models, we propose both light-weight and conventional models capable of extracting action features from video frames.
• Score: 15.915410623440874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advances in machine learning and contactless sensors have enabled the understanding complex human behaviors in a healthcare setting. In particular, several deep learning systems have been introduced to enable comprehensive analysis of neuro-developmental conditions such as Autism Spectrum Disorder (ASD). This condition affects children from their early developmental stages onwards, and diagnosis relies entirely on observing the child's behavior and detecting behavioral cues. However, the diagnosis process is time-consuming as it requires long-term behavior observation, and the scarce availability of specialists. We demonstrate the effect of a region-based computer vision system to help clinicians and parents analyze a child's behavior. For this purpose, we adopt and enhance a dataset for analyzing autism-related actions using videos of children captured in uncontrolled environments (e.g. videos collected with consumer-grade cameras, in varied environments). The data is pre-processed by detecting the target child in the video to reduce the impact of background noise. Motivated by the effectiveness of temporal convolutional models, we propose both light-weight and conventional models capable of extracting action features from video frames and classifying autism-related behaviors by analyzing the relationships between frames in a video. Through extensive evaluations on the feature extraction and learning strategies, we demonstrate that the best performance is achieved with an Inflated 3D Convnet and Multi-Stage Temporal Convolutional Networks, achieving a 0.83 Weighted F1-score for classification of the three autism-related actions, outperforming existing methods. We also propose a light-weight solution by employing the ESNet backbone within the same system, achieving competitive results of 0.71 Weighted F1-score, and enabling potential deployment on embedded systems.

Related papers

• Advanced Gesture Recognition in Autism: Integrating YOLOv7, Video Augmentation and VideoMAE for Video Analysis [9.162792034193373]
  This research work aims to identify repetitive behaviors indicative of autism by analyzing videos captured in natural settings as children engage in daily activities. The focus is on accurately categorizing real-time repetitive gestures such as spinning, head banging, and arm flapping. A key component of the proposed methodology is the use of textbfVideoMAE, a model designed to improve both spatial and temporal analysis of video data.
  arXiv  Detail & Related papers  (2024-10-12T02:55:37Z)
• Towards Child-Inclusive Clinical Video Understanding for Autism Spectrum Disorder [27.788204861041553]
  We investigate the use of foundation models across three modalities: speech, video, and text, to analyse child-focused interaction sessions. We evaluate their performance on two tasks with different information granularity: activity recognition and abnormal behavior detection.
  arXiv  Detail & Related papers  (2024-09-20T16:06:46Z)
• Enhancing Autism Spectrum Disorder Early Detection with the Parent-Child Dyads Block-Play Protocol and an Attention-enhanced GCN-xLSTM Hybrid Deep Learning Framework [6.785167067600156]
  This work proposes a novel Parent-Child Dyads Block-Play (PCB) protocol to identify behavioral patterns distinguishing ASD from typically developing toddlers. We have compiled a substantial video dataset, featuring 40 ASD and 89 TD toddlers engaged in block play with parents. This dataset exceeds previous efforts on both the scale of participants and the length of individual sessions.
  arXiv  Detail & Related papers  (2024-08-29T21:53:01Z)
• Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition [47.550391816383794]
  We introduce a novel problem of audio-visual autism behavior recognition. Social behavior recognition is an essential aspect previously omitted in AI-assisted autism screening research. We will release our dataset, code, and pre-trained models.
  arXiv  Detail & Related papers  (2024-03-22T22:52:35Z)
• Video-Based Autism Detection with Deep Learning [0.0]
  We develop a deep learning model that analyzes video clips of children reacting to sensory stimuli. Results show that our model effectively generalizes and understands key differences in the distinct movements of the children.
  arXiv  Detail & Related papers  (2024-02-26T17:45:00Z)
• Exploiting the Brain's Network Structure for Automatic Identification of ADHD Subjects [70.37277191524755]
  We show that the brain can be modeled as a functional network, and certain properties of the networks differ in ADHD subjects from control subjects. We train our classifier with 776 subjects and test on 171 subjects provided by The Neuro Bureau for the ADHD-200 challenge.
  arXiv  Detail & Related papers  (2023-06-15T16:22:57Z)
• NeuroExplainer: Fine-Grained Attention Decoding to Uncover Cortical Development Patterns of Preterm Infants [73.85768093666582]
  We propose an explainable geometric deep network dubbed NeuroExplainer. NeuroExplainer is used to uncover altered infant cortical development patterns associated with preterm birth.
  arXiv  Detail & Related papers  (2023-01-01T12:48:12Z)
• Language-Assisted Deep Learning for Autistic Behaviors Recognition [13.200025637384897]
  We show that a vision-based problem behaviors recognition system can achieve high accuracy and outperform the previous methods by a large margin. We propose a two-branch multimodal deep learning framework by incorporating the "freely available" language description for each type of problem behavior. Experimental results demonstrate that incorporating additional language supervision can bring an obvious performance boost for the autism problem behaviors recognition task.
  arXiv  Detail & Related papers  (2022-11-17T02:58:55Z)
• Muti-view Mouse Social Behaviour Recognition with Deep Graphical Model [124.26611454540813]
  Social behaviour analysis of mice is an invaluable tool to assess therapeutic efficacy of neurodegenerative diseases. Because of the potential to create rich descriptions of mouse social behaviors, the use of multi-view video recordings for rodent observations is increasingly receiving much attention. We propose a novel multiview latent-attention and dynamic discriminative model that jointly learns view-specific and view-shared sub-structures.
  arXiv  Detail & Related papers  (2020-11-04T18:09:58Z)
• Early Autism Spectrum Disorders Diagnosis Using Eye-Tracking Technology [62.997667081978825]
  Lack of money, absence of qualified specialists, and low level of trust to the correction methods are the main issues that affect the in-time diagnoses of ASD. Our team developed the algorithm that will be able to predict the chances of ASD according to the information from the gaze activity of the child.
  arXiv  Detail & Related papers  (2020-08-21T20:22:55Z)
• Retinopathy of Prematurity Stage Diagnosis Using Object Segmentation and Convolutional Neural Networks [68.96150598294072]
  Retinopathy of Prematurity (ROP) is an eye disorder primarily affecting premature infants with lower weights. It causes proliferation of vessels in the retina and could result in vision loss and, eventually, retinal detachment, leading to blindness. In recent years, there has been a significant effort to automate the diagnosis using deep learning. This paper builds upon the success of previous models and develops a novel architecture, which combines object segmentation and convolutional neural networks (CNN) Our proposed system first trains an object segmentation model to identify the demarcation line at a pixel level and adds the resulting mask as an additional "color" channel in
  arXiv  Detail & Related papers  (2020-04-03T14:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.