Script-centric behavior understanding for assisted autism spectrum disorder diagnosis

• URL: http://arxiv.org/abs/2411.09413v1
• Date: Thu, 14 Nov 2024 13:07:19 GMT
• Title: Script-centric behavior understanding for assisted autism spectrum disorder diagnosis
• Authors: Wenxing Liu, Yueran Pan, Ming Li,
• Abstract summary: This work focuses on automatically detecting Autism Spectrum Disorders (ASD) using computer vision techniques and large language models (LLMs) Our pipeline converts video content into scripts that describe the behavior of characters, leveraging the generalizability of large language models to detect ASD in a zero-shot or few-shot manner. Our method achieves an accuracy of 92.00% in diagnosing ASD in children with an average age of 24 months, surpassing the performance of supervised learning methods by 3.58% absolutely.
• Score: 6.198128116862245
• License:
• Abstract: Observing and analyzing children's social behaviors is crucial for the early diagnosis of Autism Spectrum Disorders (ASD). This work focuses on automatically detecting ASD using computer vision techniques and large language models (LLMs). Existing methods typically rely on supervised learning. However, the scarcity of ASD diagnostic datasets and the lack of interpretability in diagnostic results significantly limits its clinical application. To address these challenges, we introduce a novel unsupervised approach based on script-centric behavior understanding. Our pipeline converts video content into scripts that describe the behavior of characters, leveraging the generalizability of large language models to detect ASD in a zero-shot or few-shot manner. Specifically, we propose a scripts transcription module for multimodal behavior data textualization and a domain prompts module to bridge LLMs. Our method achieves an accuracy of 92.00\% in diagnosing ASD in children with an average age of 24 months, surpassing the performance of supervised learning methods by 3.58\% absolutely. Extensive experiments confirm the effectiveness of our approach and suggest its potential for advancing ASD research through LLMs.

Related papers

• Explainable AI for Autism Diagnosis: Identifying Critical Brain Regions Using fMRI Data [0.29687381456163997]
  Early diagnosis and intervention for Autism Spectrum Disorder (ASD) has been shown to significantly improve the quality of life of autistic individuals. There is a need for objective biomarkers of ASD which can help improve diagnostic accuracy. Deep learning (DL) has achieved outstanding performance in diagnosing diseases and conditions from medical imaging data. This research aims to improve the accuracy and interpretability of ASD diagnosis by creating a DL model that can not only accurately classify ASD but also provide explainable insights into its working.
  arXiv  Detail & Related papers  (2024-09-19T23:08:09Z)
• Diagnostic Reasoning in Natural Language: Computational Model and Application [68.47402386668846]
  We investigate diagnostic abductive reasoning (DAR) in the context of language-grounded tasks (NL-DAR) We propose a novel modeling framework for NL-DAR based on Pearl's structural causal models. We use the resulting dataset to investigate the human decision-making process in NL-DAR.
  arXiv  Detail & Related papers  (2024-09-09T06:55:37Z)
• Localizing Moments of Actions in Untrimmed Videos of Infants with Autism Spectrum Disorder [5.2289135066938375]
  We introduce a self-attention based TAL model designed to identify ASD-related behaviors in infant videos. This study is the first to conduct end-to-end temporal action localization in untrimmed videos of infants with ASD. We achieve 70% accuracy for look face, 79% accuracy for look object, 72% for smile and 65% for vocalization.
  arXiv  Detail & Related papers  (2024-04-08T20:31:27Z)
• Enhancing ASD detection accuracy: a combined approach of machine learning and deep learning models with natural language processing [0.0]
  Our study explored the use of artificial intelligence (AI) to diagnose autism spectrum disorder (ASD) It focused on machine learning (ML) and deep learning (DL) to detect ASD from text inputs on social media. Our AI models showed high accuracy, with an 88% success rate in identifying texts from individuals with ASD.
  arXiv  Detail & Related papers  (2024-03-06T09:57:42Z)
• Introducing SSBD+ Dataset with a Convolutional Pipeline for detecting Self-Stimulatory Behaviours in Children using raw videos [1.1874952582465603]
  The authors propose a novel pipelined deep learning architecture to detect certain self-stimulatory behaviors that help in the diagnosis of autism spectrum disorder (ASD) An overall accuracy of around 81% was achieved from the proposed pipeline model that is targeted for real-time and hands-free automated diagnosis.
  arXiv  Detail & Related papers  (2023-11-25T16:57:24Z)
• Leveraging Pretrained Representations with Task-related Keywords for Alzheimer's Disease Detection [69.53626024091076]
  Alzheimer's disease (AD) is particularly prominent in older adults. Recent advances in pre-trained models motivate AD detection modeling to shift from low-level features to high-level representations. This paper presents several efficient methods to extract better AD-related cues from high-level acoustic and linguistic features.
  arXiv  Detail & Related papers  (2023-03-14T16:03:28Z)
• Comparison of Probabilistic Deep Learning Methods for Autism Detection [0.0]
  Autism Spectrum Disorder (ASD) is one neuro developmental disorder that is now widespread in the world. Early detection of the disorder helps in the onset treatment and helps one to lead a normal life.
  arXiv  Detail & Related papers  (2023-03-09T17:49:37Z)
• Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection [70.86672569101536]
  Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and to delay further progression. This paper investigates the use of prompt-based fine-tuning of PLMs that consistently uses AD classification errors as the training objective function.
  arXiv  Detail & Related papers  (2022-10-29T09:18:41Z)
• "My nose is running.""Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics [80.55587329326046]
  We propose a more interpretable decision process to implement the dialogue manager of DSMD. We devise a model with highly transparent components to conduct the inference. Experiments show that our method obtains 7.7%, 10.0%, 3.0% absolute improvement in diagnosis accuracy.
  arXiv  Detail & Related papers  (2022-04-29T09:02:23Z)
• Early Autism Spectrum Disorders Diagnosis Using Eye-Tracking Technology [62.997667081978825]
  Lack of money, absence of qualified specialists, and low level of trust to the correction methods are the main issues that affect the in-time diagnoses of ASD. Our team developed the algorithm that will be able to predict the chances of ASD according to the information from the gaze activity of the child.
  arXiv  Detail & Related papers  (2020-08-21T20:22:55Z)
• Detecting Parkinsonian Tremor from IMU Data Collected In-The-Wild using Deep Multiple-Instance Learning [59.74684475991192]
  Parkinson's Disease (PD) is a slowly evolving neuro-logical disease that affects about 1% of the population above 60 years old. PD symptoms include tremor, rigidity and braykinesia. We present a method for automatically identifying tremorous episodes related to PD, based on IMU signals captured via a smartphone device.
  arXiv  Detail & Related papers  (2020-05-06T09:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.