Improving Autism Detection with Multimodal Behavioral Analysis

• URL: http://arxiv.org/abs/2509.21352v1
• Date: Fri, 19 Sep 2025 13:08:50 GMT
• Title: Improving Autism Detection with Multimodal Behavioral Analysis
• Authors: William Saakyan, Matthias Norden, Lola Eversmann, Simon Kirsch, Muyu Lin, Simon Guendelman, Isabel Dziobek, Hanna Drimalla,
• Abstract summary: We conduct a multimodal analysis of facial expressions, voice prosody, head motion, heart rate variability (HRV) and gaze behavior.<n>We achieve a classification accuracy of 74%, demonstrating the effectiveness of integrating behavioral markers across multiple modalities.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Due to the complex and resource-intensive nature of diagnosing Autism Spectrum Condition (ASC), several computer-aided diagnostic support methods have been proposed to detect autism by analyzing behavioral cues in patient video data. While these models show promising results on some datasets, they struggle with poor gaze feature performance and lack of real-world generalizability. To tackle these challenges, we analyze a standardized video dataset comprising 168 participants with ASC (46% female) and 157 non-autistic participants (46% female), making it, to our knowledge, the largest and most balanced dataset available. We conduct a multimodal analysis of facial expressions, voice prosody, head motion, heart rate variability (HRV), and gaze behavior. To address the limitations of prior gaze models, we introduce novel statistical descriptors that quantify variability in eye gaze angles, improving gaze-based classification accuracy from 64% to 69% and aligning computational findings with clinical research on gaze aversion in ASC. Using late fusion, we achieve a classification accuracy of 74%, demonstrating the effectiveness of integrating behavioral markers across multiple modalities. Our findings highlight the potential for scalable, video-based screening tools to support autism assessment.

Related papers

• Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis [42.60508892284938]
  We present a large-scale, multi-center benchmark of deep unsupervised anomaly detection for brain imaging.<n>We tested 2,221 T1w and 1,262 T2w scans spanning healthy datasets and diverse clinical cohorts.<n>Our benchmark establishes a transparent foundation for future research and highlights priorities for clinical translation.
  arXiv  Detail & Related papers  (2025-12-01T11:03:27Z)
• Evolving Diagnostic Agents in a Virtual Clinical Environment [75.59389103511559]
  We present a framework for training large language models (LLMs) as diagnostic agents with reinforcement learning.<n>Our method acquires diagnostic strategies through interactive exploration and outcome-based feedback.<n>DiagAgent significantly outperforms 10 state-of-the-art LLMs, including DeepSeek-v3 and GPT-4o.
  arXiv  Detail & Related papers  (2025-10-28T17:19:47Z)
• An Explainable Hybrid AI Framework for Enhanced Tuberculosis and Symptom Detection [55.35661671061754]
  Tuberculosis remains a critical global health issue, particularly in resource-limited and remote areas.<n>We propose a framework which enhances disease and symptom detection on chest X-rays by integrating two supervised heads and a self-supervised head.<n>Our model achieves an accuracy of 98.85% for distinguishing between COVID-19, tuberculosis, and normal cases, and a macro-F1 score of 90.09% for multilabel symptom detection.
  arXiv  Detail & Related papers  (2025-10-21T17:18:55Z)
• Bi-cephalic self-attended model to classify Parkinson's disease patients with freezing of gait [2.8658669653771334]
  This study aims to develop an objective, data-driven, and multi-modal classification model to detect gait dysfunction in PD patients.<n>Features extracted from resting-state EEG and descriptive variables were used to train a novel Bi-cephalic Self-Attention Model (BiSAM)
  arXiv  Detail & Related papers  (2025-07-28T14:16:01Z)
• MIL vs. Aggregation: Evaluating Patient-Level Survival Prediction Strategies Using Graph-Based Learning [52.231128973251124]
  We compare various strategies for predicting survival at the WSI and patient level.<n>The former treats each WSI as an independent sample, mimicking the strategy adopted in other works.<n>The latter comprises methods to either aggregate the predictions of the several WSIs or automatically identify the most relevant slide.
  arXiv  Detail & Related papers  (2025-03-29T11:14:02Z)
• Congenital Heart Disease Classification Using Phonocardiograms: A Scalable Screening Tool for Diverse Environments [34.10187730651477]
  Congenital heart disease (CHD) is a critical condition that demands early detection.<n>This study presents a deep learning model designed to detect CHD using phonocardiogram (PCG) signals.<n>We evaluated our model on several datasets, including the primary dataset from Bangladesh.
  arXiv  Detail & Related papers  (2025-03-28T05:47:44Z)
• Urinary Tract Infection Detection in Digital Remote Monitoring: Strategies for Managing Participant-Specific Prediction Complexity [43.108040967674185]
  Urinary tract infections (UTIs) are a significant health concern, particularly for people living with dementia (PLWD)<n>This study builds on previous work that utilised machine learning (ML) to detect UTIs in PLWD.
  arXiv  Detail & Related papers  (2025-02-18T12:01:55Z)
• Self-Explaining Hypergraph Neural Networks for Diagnosis Prediction [45.89562183034469]
  Existing deep learning diagnosis prediction models with intrinsic interpretability often assign attention weights to every past diagnosis or hospital visit.<n>We introduce SHy, a self-explaining hypergraph neural network model, designed to offer personalized, concise and faithful explanations.<n> SHy captures higher-order disease interactions and extracts distinct temporal phenotypes as personalized explanations.
  arXiv  Detail & Related papers  (2025-02-15T06:33:02Z)
• Towards Equitable ASD Diagnostics: A Comparative Study of Machine and Deep Learning Models Using Behavioral and Facial Data [2.6353853440763113]
  Autism Spectrum Disorder (ASD) is often underdiagnosed in females due to gender-specific symptom differences. This study evaluates machine learning models, particularly Random Forest and convolutional neural networks, for enhancing ASD diagnosis.
  arXiv  Detail & Related papers  (2024-11-08T05:26:04Z)
• Ensemble Modeling of Multiple Physical Indicators to Dynamically Phenotype Autism Spectrum Disorder [3.6630139570443996]
  We provide a dataset for training computer vision models to detect Autism Spectrum Disorder (ASD)-related phenotypic markers. We trained individual LSTM-based models using eye gaze, head positions, and facial landmarks as input features, achieving test AUCs of 86%, 67%, and 78%.
  arXiv  Detail & Related papers  (2024-08-23T17:55:58Z)
• MADE-for-ASD: A Multi-Atlas Deep Ensemble Network for Diagnosing Autism Spectrum Disorder [4.7377709803078325]
  This paper bridges the gap between traditional, time-consuming diagnostic methods and potential automated solutions. We propose a multi-atlas deep ensemble network, MADE-for-ASD, that integrates multiple atlases of the brain's functional magnetic resonance imaging (fMRI) data. Our approach integrates demographic information into the prediction workflow, which enhances ASD diagnosis performance.
  arXiv  Detail & Related papers  (2024-07-09T17:49:23Z)
• Methodology and Real-World Applications of Dynamic Uncertain Causality Graph for Clinical Diagnosis with Explainability and Invariance [41.373856519548404]
  Dynamic Uncertain Causality Graph (DUCG) approach is causality-driven, explainable, and invariant across different application scenarios. 46 DUCG models covering 54 chief complaints were constructed. Over one million real diagnosis cases have been performed, with only 17 incorrect diagnoses identified.
  arXiv  Detail & Related papers  (2024-06-09T11:37:45Z)
• Diagnosing Human-object Interaction Detectors [42.283857276076596]
  We introduce a diagnosis toolbox to provide detailed quantitative break-down analysis of HOI detection models. We analyze eight state-of-the-art HOI detection models and provide valuable diagnosis insights to foster future research.
  arXiv  Detail & Related papers  (2023-08-16T17:39:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.