Multimodal Ensemble with Conditional Feature Fusion for Dysgraphia Diagnosis in Children from Handwriting Samples

• URL: http://arxiv.org/abs/2408.13754v1
• Date: Sun, 25 Aug 2024 07:42:54 GMT
• Title: Multimodal Ensemble with Conditional Feature Fusion for Dysgraphia Diagnosis in Children from Handwriting Samples
• Authors: Jayakanth Kunhoth, Somaya Al-Maadeed, Moutaz Saleh, Younes Akbari,
• Abstract summary: Developmental dysgraphia is a neurological disorder that hinders children's writing skills. We propose a novel multimodal machine learning approach utilizing both online and offline handwriting data. Our approach achieves an accuracy of 88.8%, outperforming SVMs for single modalities by 12-14%.
• Score: 0.7671590907056449
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Developmental dysgraphia is a neurological disorder that hinders children's writing skills. In recent years, researchers have increasingly explored machine learning methods to support the diagnosis of dysgraphia based on offline and online handwriting. In most previous studies, the two types of handwriting have been analysed separately, which does not necessarily lead to promising results. In this way, the relationship between online and offline data cannot be explored. To address this limitation, we propose a novel multimodal machine learning approach utilizing both online and offline handwriting data. We created a new dataset by transforming an existing online handwritten dataset, generating corresponding offline handwriting images. We considered only different types of word data (simple word, pseudoword & difficult word) in our multimodal analysis. We trained SVM and XGBoost classifiers separately on online and offline features as well as implemented multimodal feature fusion and soft-voted ensemble. Furthermore, we proposed a novel ensemble with conditional feature fusion method which intelligently combines predictions from online and offline classifiers, selectively incorporating feature fusion when confidence scores fall below a threshold. Our novel approach achieves an accuracy of 88.8%, outperforming SVMs for single modalities by 12-14%, existing methods by 8-9%, and traditional multimodal approaches (soft-vote ensemble and feature fusion) by 3% and 5%, respectively. Our methodology contributes to the development of accurate and efficient dysgraphia diagnosis tools, requiring only a single instance of multimodal word/pseudoword data to determine the handwriting impairment. This work highlights the potential of multimodal learning in enhancing dysgraphia diagnosis, paving the way for accessible and practical diagnostic tools.

Related papers

• Explainable AI in Handwriting Detection for Dyslexia Using Transfer Learning [0.0]
  We propose an explainable AI (XAI) framework for dyslexia detection through handwriting analysis. Our approach surpasses state-of-the-art methods, achieving a test accuracy of 0.9958. This framework not only improves diagnostic accuracy but also fosters trust and understanding among educators, clinicians, and parents.
  arXiv  Detail & Related papers  (2024-10-18T11:14:54Z)
• Hybrid Transformer for Early Alzheimer's Detection: Integration of Handwriting-Based 2D Images and 1D Signal Features [2.606124959056959]
  Alzheimer's Disease (AD) is a prevalent neurodegenerative condition where early detection is vital. Handwriting, often affected early in AD, offers a non-invasive and cost-effective way to capture subtle motor changes. We propose a learnable multimodal hybrid attention model that integrates simultaneously 2D handwriting images with 1D dynamic handwriting signals. Our model achieved state-of-the-art performance on the DARWIN dataset, with an F1-score of 90.32% and accuracy of 90.91% in Task 8 ('L' writing)
  arXiv  Detail & Related papers  (2024-10-14T14:26:52Z)
• DiffusionPen: Towards Controlling the Style of Handwritten Text Generation [7.398476020996681]
  DiffusionPen (DiffPen) is a 5-shot style handwritten text generation approach based on Latent Diffusion Models. Our approach captures both textual and stylistic characteristics of seen and unseen words and styles, generating realistic handwritten samples. Our method outperforms existing methods qualitatively and quantitatively, and its additional generated data can improve the performance of Handwriting Text Recognition (HTR) systems.
  arXiv  Detail & Related papers  (2024-09-09T20:58:25Z)
• Retrieval is Accurate Generation [99.24267226311157]
  We introduce a novel method that selects context-aware phrases from a collection of supporting documents. Our model achieves the best performance and the lowest latency among several retrieval-augmented baselines.
  arXiv  Detail & Related papers  (2024-02-27T14:16:19Z)
• DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text [82.5469544192645]
  We propose a novel training-free detection strategy called Divergent N-Gram Analysis (DNA-GPT) By analyzing the differences between the original and new remaining parts through N-gram analysis, we unveil significant discrepancies between the distribution of machine-generated text and human-written text. Results show that our zero-shot approach exhibits state-of-the-art performance in distinguishing between human and GPT-generated text.
  arXiv  Detail & Related papers  (2023-05-27T03:58:29Z)
• Meta Learning for Few-Shot Medical Text Classification [1.2617078020344619]
  We investigate the use of meta-learning and robustness techniques on a broad corpus of benchmark text and medical data. We find that meta-learning on text is a suitable framework for text-based data, providing better data efficiency and comparable performance to few-shot language models.
  arXiv  Detail & Related papers  (2022-12-03T06:46:52Z)
• Recognizing Handwriting Styles in a Historical Scanned Document Using Unsupervised Fuzzy Clustering [0.0]
  Unique handwriting styles may be dissimilar in a blend of several factors including character size, stroke width, loops, ductus, slant angles, and cursive ligatures. Previous work on labeled data with Hidden Markov models, support vector machines, and semi-supervised recurrent neural networks have provided moderate to high success. In this study, we successfully detect hand shifts in a historical manuscript through fuzzy soft clustering in combination with linear principal component analysis.
  arXiv  Detail & Related papers  (2022-10-30T09:07:51Z)
• $\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation [65.29170569821093]
  parallel text generation has received widespread attention due to its success in generation efficiency. In this paper, we propose $textitlatent$-GLAT, which employs the discrete latent variables to capture word categorical information. Experiment results show that our method outperforms strong baselines without the help of an autoregressive model.
  arXiv  Detail & Related papers  (2022-04-05T07:34:12Z)
• Letter-level Online Writer Identification [86.13203975836556]
  We focus on a novel problem, letter-level online writer-id, which requires only a few trajectories of written letters as identification cues. A main challenge is that a person often writes a letter in different styles from time to time. We refer to this problem as the variance of online writing styles (Var-O-Styles)
  arXiv  Detail & Related papers  (2021-12-06T07:21:53Z)
• Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
  Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
  arXiv  Detail & Related papers  (2021-10-04T08:51:36Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.