Understanding Dementia Speech Alignment with Diffusion-Based Image Generation

• URL: http://arxiv.org/abs/2508.09385v1
• Date: Tue, 12 Aug 2025 23:00:36 GMT
• Title: Understanding Dementia Speech Alignment with Diffusion-Based Image Generation
• Abstract summary: We show that dementia detection is possible from generated images alone achieving 75% accuracy on the ADReSS dataset.<n>We then leverage explainability methods to show which parts of the language contribute to the detection.
• Score: 2.534053759586253
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-image models generate highly realistic images based on natural language descriptions and millions of users use them to create and share images online. While it is expected that such models can align input text and generated image in the same latent space little has been done to understand whether this alignment is possible between pathological speech and generated images. In this work, we examine the ability of such models to align dementia-related speech information with the generated images and develop methods to explain this alignment. Surprisingly, we found that dementia detection is possible from generated images alone achieving 75% accuracy on the ADReSS dataset. We then leverage explainability methods to show which parts of the language contribute to the detection.

Related papers

• Plasticine: A Traceable Diffusion Model for Medical Image Translation [79.39689106440389]
  We propose Plasticine, to the best of our knowledge, the first end-to-end image-to-image translation framework explicitly designed with traceability as a core objective.<n>Our method combines intensity translation and spatial transformation within a denoising diffusion framework.<n>This design enables the generation of synthetic images with interpretable intensity transitions and spatially coherent deformations, supporting pixel-wise traceability throughout the translation process.
  arXiv  Detail & Related papers  (2025-12-20T18:01:57Z)
• Explainable Detection of AI-Generated Images with Artifact Localization Using Faster-Than-Lies and Vision-Language Models for Edge Devices [0.0]
  We present an explainable image authenticity detection system that combines a lightweight convolutional classifier ("Faster-Than-Lies") with a Vision-Language Model (Qwen2-VL-7B)<n>Our model achieves 96.5% accuracy on the extended CiFAKE dataset augmented with adversarial perturbations, enabling deployment on local or edge devices.<n>This work highlights the feasibility of combining visual and linguistic reasoning for interpretable authenticity detection in low-resolution imagery and outlines potential cross-domain applications in forensics, industrial inspection, and social media moderation.
  arXiv  Detail & Related papers  (2025-10-27T19:01:24Z)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation [120.23172120151821]
  We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models.<n>We introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences.<n>We propose a new metric, Visual Semantic Matching, that quantifies visual inconsistencies in subject-driven image generation.
  arXiv  Detail & Related papers  (2025-09-26T07:11:55Z)
• RL4Med-DDPO: Reinforcement Learning for Controlled Guidance Towards Diverse Medical Image Generation using Vision-Language Foundation Models [0.7165255458140439]
  Vision-Language Foundation Models (VLFM) have shown a tremendous increase in performance in terms of generating high-resolution, photorealistic natural images.<n>We propose a multi-stage architecture where a pre-trained VLFM provides a cursory semantic understanding, while a reinforcement learning algorithm refines the alignment through an iterative process.<n>The reward signal is designed to align the semantic information of the text with synthesized images.
  arXiv  Detail & Related papers  (2025-03-20T01:51:05Z)
• ISImed: A Framework for Self-Supervised Learning using Intrinsic Spatial Information in Medical Images [0.0]
  This paper demonstrates that spatial information can be used to learn interpretable representations in medical images using Self-Supervised Learning (SSL) We establish a self-supervised objective that creates a latent representation capable of capturing its location in the physical realm. We show that our method can efficiently learn representations that capture the underlying structure of the data and can be used to transfer to a downstream classification task.
  arXiv  Detail & Related papers  (2024-10-22T12:21:39Z)
• Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning [64.1316997189396]
  We present a novel language-tied self-supervised learning framework, Hierarchical Language-tied Self-Supervision (HLSS) for histopathology images. Our resulting model achieves state-of-the-art performance on two medical imaging benchmarks, OpenSRH and TCGA datasets.
  arXiv  Detail & Related papers  (2024-03-21T17:58:56Z)
• Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment [64.49170817854942]
  We present a method to provide detailed explanation of detected misalignments between text-image pairs. We leverage large language models and visual grounding models to automatically construct a training set that holds plausible captions for a given image. We also publish a new human curated test set comprising ground-truth textual and visual misalignment annotations.
  arXiv  Detail & Related papers  (2023-12-05T20:07:34Z)
• Improving Compositional Text-to-image Generation with Large Vision-Language Models [26.202725136839632]
  compositional text-to-image models frequently encounter difficulties in generating high-quality images that align with input texts. We employ large vision-language models (LVLMs) for multi-dimensional assessment of the alignment between generated images and their corresponding input texts. Our experimental results validate that the proposed methodology significantly improves text-image alignment in compositional image generation.
  arXiv  Detail & Related papers  (2023-10-10T05:09:05Z)
• Evaluating Picture Description Speech for Dementia Detection using Image-text Alignment [10.008388878255538]
  We propose the first dementia detection models that take both the picture and the description texts as inputs. We observe the difference between dementia and healthy samples in terms of the text's relevance to the picture and the focused area of the picture. We propose three advanced models that pre-processed the samples based on their relevance to the picture, sub-image, and focused areas.
  arXiv  Detail & Related papers  (2023-08-11T08:42:37Z)
• Improving Generalization of Image Captioning with Unsupervised Prompt Learning [63.26197177542422]
  Generalization of Image Captioning (GeneIC) learns a domain-specific prompt vector for the target domain without requiring annotated data. GeneIC aligns visual and language modalities with a pre-trained Contrastive Language-Image Pre-Training (CLIP) model.
  arXiv  Detail & Related papers  (2023-08-05T12:27:01Z)
• Zero-shot spatial layout conditioning for text-to-image diffusion models [52.24744018240424]
  Large-scale text-to-image diffusion models have significantly improved the state of the art in generative image modelling. We consider image generation from text associated with segments on the image canvas, which combines an intuitive natural language interface with precise spatial control over the generated content. We propose ZestGuide, a zero-shot segmentation guidance approach that can be plugged into pre-trained text-to-image diffusion models.
  arXiv  Detail & Related papers  (2023-06-23T19:24:48Z)
• Structural-analogy from a Single Image Pair [118.61885732829117]
  In this paper, we explore the capabilities of neural networks to understand image structure given only a single pair of images, A and B. We generate an image that keeps the appearance and style of B, but has a structural arrangement that corresponds to A. Our method can be used to generate high quality imagery in other conditional generation tasks utilizing images A and B only.
  arXiv  Detail & Related papers  (2020-04-05T14:51:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.