Inducing Dyslexia in Vision Language Models

• URL: http://arxiv.org/abs/2509.24597v2
• Date: Tue, 30 Sep 2025 09:36:07 GMT
• Title: Inducing Dyslexia in Vision Language Models
• Authors: Melika Honarmand, Ayati Sharma, Badr AlKhamissi, Johannes Mehrer, Martin Schrimpf,
• Abstract summary: We use large-scale vision-language models to simulate dyslexia.<n>We identify visual-word-form-selective units within VLMs and demonstrate that targeted ablation of these units leads to selective impairments in reading tasks.
• Score: 6.080220062810505
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Dyslexia, a neurodevelopmental disorder characterized by persistent reading difficulties, is often linked to reduced activity of the visual word form area in the ventral occipito-temporal cortex. Traditional approaches to studying dyslexia, such as behavioral and neuroimaging methods, have provided valuable insights but remain limited in their ability to test causal hypotheses about the underlying mechanisms of reading impairments. In this study, we use large-scale vision-language models (VLMs) to simulate dyslexia by functionally identifying and perturbing artificial analogues of word processing. Using stimuli from cognitive neuroscience, we identify visual-word-form-selective units within VLMs and demonstrate that targeted ablation of these units, unlike ablation of random units, leads to selective impairments in reading tasks while general visual and language comprehension abilities remain intact. In particular, the resulting model matches dyslexic humans' phonological deficits without a significant change in orthographic processing. Taken together, our modeling results replicate key characteristics of dyslexia and establish a computational framework for investigating reading disorders.

Related papers

• Quantifying the Effects of Word Length, Frequency, and Predictability on Dyslexia [0.0]
  Using eye-tracking aligned to word-level features, we model how each feature influences dyslexic time costs.<n>We find that all three features robustly change reading times in both typical and dyslexic readers.
  arXiv  Detail & Related papers  (2025-10-28T17:15:31Z)
• Devising a Set of Compact and Explainable Spoken Language Feature for Screening Alzheimer's Disease [52.46922921214341]
  Alzheimer's disease (AD) has become one of the most significant health challenges in an aging society.<n>We devised an explainable and effective feature set that leverages the visual capabilities of a large language model (LLM) and the Term Frequency-Inverse Document Frequency (TF-IDF) model.<n>Our new features can be well explained and interpreted step by step which enhance the interpretability of automatic AD screening.
  arXiv  Detail & Related papers  (2024-11-28T05:23:22Z)
• Generative causal testing to bridge data-driven models and scientific theories in language neuroscience [82.995061475971]
  We present generative causal testing (GCT), a framework for generating concise explanations of language selectivity in the brain.<n>We show that GCT can dissect fine-grained differences between brain areas with similar functional selectivity.
  arXiv  Detail & Related papers  (2024-10-01T15:57:48Z)
• Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation [63.064204206220936]
  Foundational Large Language Models (LLMs) have changed the way we perceive technology. They have been shown to excel in tasks ranging from poem writing to coding to essay generation and puzzle solving. With the incorporation of image generation capability, they have become more comprehensive and versatile AI tools. Currently identified flaws include hallucination, biases, and bypassing restricted commands to generate harmful content.
  arXiv  Detail & Related papers  (2024-08-27T14:40:16Z)
• Handwriting Anomalies and Learning Disabilities through Recurrent Neural Networks and Geometric Pattern Analysis [0.0]
  This study utilizes advanced geometrical patterns and recurrent neural networks (RNN) to identify handwriting anomalies indicative of dyslexia and dysgraphia.<n>Handwriting is first standardized, followed by feature extraction that focuses on baseline deviations, letter connectivity, stroke thickness, and other anomalies.<n>Initial results demonstrate the ability of this RNN model to achieve state-of-art performance on combined dyslexia and dysgraphia detection.
  arXiv  Detail & Related papers  (2024-05-12T10:10:13Z)
• Integrating large language models and active inference to understand eye movements in reading and dyslexia [0.0]
  We present a novel computational model employing hierarchical active inference to simulate reading and eye movements.<n>Our model can potentially aid in understanding how maladaptive predictive processing can produce reading deficits associated with dyslexia.
  arXiv  Detail & Related papers  (2023-08-09T13:16:30Z)
• Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context [87.31930367845125]
  We trained a lexical language model, Glove, and a supra-lexical language model, GPT-2, on a text corpus. We then assessed to what extent these information-restricted models were able to predict the time-courses of fMRI signal of humans listening to naturalistic text. Our analyses show that, while most brain regions involved in language are sensitive to both syntactic and semantic variables, the relative magnitudes of these effects vary a lot across these regions.
  arXiv  Detail & Related papers  (2023-02-28T08:16:18Z)
• Eye-tracking based classification of Mandarin Chinese readers with and without dyslexia using neural sequence models [7.639036130018945]
  We propose two simple sequence models that process eye movements on the entire stimulus without the need of aggregating features across the sentence. We incorporate the linguistic stimulus into the model in two ways -- contextualized word embeddings and manually extracted linguistic features. Our results show that (i) even for a logographic script such as Chinese, sequence models are able to classify dyslexia on eye gaze sequences, reaching state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-10-18T12:57:30Z)
• CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals [60.921888445317705]
  We propose a CogAlign approach to integrate cognitive language processing signals into natural language processing models. We show that CogAlign achieves significant improvements with multiple cognitive features over state-of-the-art models on public datasets.
  arXiv  Detail & Related papers  (2021-06-10T07:10:25Z)
• On-the-Fly Attention Modularization for Neural Generation [54.912042110885366]
  We show that generated text is repetitive, generic, self-inconsistent, and lacking commonsense. Our findings motivate on-the-fly attention modularization, a simple but effective method for injecting inductive biases into attention during inference.
  arXiv  Detail & Related papers  (2021-01-02T05:16:46Z)
• A Tale of Two Perplexities: Sensitivity of Neural Language Models to Lexical Retrieval Deficits in Dementia of the Alzheimer's Type [10.665308703417665]
  In recent years there has been a burgeoning interest in the use of computational methods to distinguish between elicited speech samples produced by patients with dementia, and those from healthy controls. The difference between perplexity estimates from two neural language models (LMs) has been shown to produce state-of-the-art performance. We find that perplexity of neural LMs is strongly and differentially associated with lexical frequency, and that a mixture model resulting from interpolating control and dementia LMs improves upon the current state-of-the-art for models trained on transcript text exclusively.
  arXiv  Detail & Related papers  (2020-05-07T16:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.