Can Large Language Models Imitate Human Speech for Clinical Assessment? LLM-Driven Data Augmentation for Cognitive Score Prediction

• URL: http://arxiv.org/abs/2605.16077v1
• Date: Fri, 15 May 2026 15:39:22 GMT
• Title: Can Large Language Models Imitate Human Speech for Clinical Assessment? LLM-Driven Data Augmentation for Cognitive Score Prediction
• Abstract summary: We propose a large language model (LLM)-driven data augmentation framework to improve the prediction of cognitive scores from speech.<n> Experiments are conducted on a Japanese corpus in which each participant provides both a spontaneous oral narrative and a written response to the same clinical prompt.
• Score: 6.724673957306738
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate assessment of cognitive decline from spontaneous speech remains challenging due to limited dataset size and class imbalance. In this work, we propose a large language model (LLM)-driven data augmentation framework to improve the prediction of cognitive scores from speech. Experiments are conducted on a Japanese corpus in which each participant provides both a spontaneous oral narrative and a written response to the same clinical prompt. The written responses serve as semantic anchors to generate multiple oral-like monologues in different styles using GPT-5. We then predict Hasegawa Dementia Scale scores, a widely used cognitive screening tool in Japan, using a Partial Least Squares regression model trained on Sentence-BERT speech embeddings. We investigate two augmentation strategies: random class-balanced selection, which yields moderate but unstable improvements, and similarity-guided class-balanced selection. The latter prioritizes semantically close synthetic samples, leading to more consistent improvements and substantially reducing prediction error for minority low-score participants while maintaining performance for the majority group. Overall, our findings demonstrate the potential of semantically guided LLM-driven augmentation as a principled approach for addressing class imbalance and improving data efficiency in clinical speech analysis.

Related papers

• On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation [88.77441715819366]
  Generative spoken language models pretrained on large-scale raw audio can continue a speech prompt with appropriate content.<n>We propose a variety of likelihood- and generative-based evaluation methods that serve in place of naive global token perplexity.
  arXiv  Detail & Related papers  (2026-01-09T22:01:56Z)
• Speech-Based Cognitive Screening: A Systematic Evaluation of LLM Adaptation Strategies [1.6807214708043468]
  Over half of US adults with Alzheimer disease and related dementias remain undiagnosed.<n>We compared large language model adaptation strategies for dementia detection using the DementiaBank speech corpus.
  arXiv  Detail & Related papers  (2025-08-24T13:15:28Z)
• CogBench: A Large Language Model Benchmark for Multilingual Speech-Based Cognitive Impairment Assessment [23.1730341293796]
  We propose CogBench, the first benchmark designed to evaluate the cross-lingual and cross-site generalizability of large language models for speech-based cognitive impairment assessment.<n>Our results show that conventional deep learning models degrade substantially when transferred across domains.<n>Our findings offer a critical step toward building clinically useful and linguistically robust speech-based cognitive assessment tools.
  arXiv  Detail & Related papers  (2025-08-05T12:06:16Z)
• Automated evaluation of children's speech fluency for low-resource languages [8.918459083715149]
  This paper proposes a system to automatically assess fluency by combining a fine-tuned multilingual ASR model and an objective metrics extraction stage.<n>We evaluate the proposed system on a dataset of children's speech in two low-resource languages, Tamil and Malay.
  arXiv  Detail & Related papers  (2025-05-26T08:25:50Z)
• Bigger But Not Better: Small Neural Language Models Outperform Large Language Models in Detection of Thought Disorder [7.585589727435719]
  We investigate whether smaller neural language models can serve as effective alternatives for detecting positive formal thought disorder.<n>Surprisingly, our results show that smaller models are more sensitive to linguistic differences associated with formal thought disorder than their larger counterparts.
  arXiv  Detail & Related papers  (2025-03-25T22:55:58Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• Automatically measuring speech fluency in people with aphasia: first achievements using read-speech data [55.84746218227712]
  This study aims at assessing the relevance of a signalprocessingalgorithm, initially developed in the field of language acquisition, for the automatic measurement of speech fluency.
  arXiv  Detail & Related papers  (2023-08-09T07:51:40Z)
• LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech [67.88748572167309]
  We present LDNet, a unified framework for mean opinion score (MOS) prediction. We propose two inference methods that provide more stable results and efficient computation.
  arXiv  Detail & Related papers  (2021-10-18T08:52:31Z)
• Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation [98.83925811122795]
  We propose to improve the sampling procedure by selecting the most informative monolingual sentences to complement the parallel data. We compute the uncertainty of monolingual sentences using the bilingual dictionary extracted from the parallel data. Experimental results on large-scale WMT English$Rightarrow$German and English$Rightarrow$Chinese datasets demonstrate the effectiveness of the proposed approach.
  arXiv  Detail & Related papers  (2021-06-02T05:01:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.