Fine-Tuning Open-Weight Language Models to Deliver Cognitive Behavioral Therapy for Depression: A Feasibility Study

• URL: http://arxiv.org/abs/2412.00251v2
• Date: Sat, 20 Sep 2025 21:29:08 GMT
• Title: Fine-Tuning Open-Weight Language Models to Deliver Cognitive Behavioral Therapy for Depression: A Feasibility Study
• Abstract summary: Cognitive Behavioral Therapy (CBT) is an evidence-based treatment for Major Depressive Disorder.<n>There exist significant barriers to individuals accessing CBT, including cost, scarcity of therapists and stigma.<n>This study explores the feasibility of fine-tuning small open language models (LLMs) to deliver CBT for depression.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cognitive Behavioral Therapy (CBT) is a well-established, evidence-based treatment for Major Depressive Disorder. Unfortunately, there exist significant barriers to individuals accessing CBT, including cost, scarcity of therapists and stigma. This study explores the feasibility of fine-tuning small open weight large language models (LLMs) to deliver CBT for depression. Using synthetic CBT transcripts generated by the Nous Research fine-tune of Llama 3.1 405b, we fine-tuned three models: Mistral 7b v0.3, Qwen 2.5 7b, and Llama 3.1 8b. CBT fidelity was evaluated through a modified Cognitive Therapy Rating Scale (CTRS). All fine-tuned models were compared against each other, as well as their instruct-tuned variants. Simulated patient transcripts were generated for the purpose of evaluating model performance, with the instruct and CBT-tuned models acting as the therapist and DeepSeek-V2.5 acting as the patient. These simulated transcripts were evaluated on a modified CTRS by Gemini 1.5 Pro-002. Our findings demonstrated that the CBT-tuned models significantly outperformed their instruct-tuned counterparts, with an average improvement of 11.33 points (p < 0.001) on total CTRS score. Llama 3.1 8b had the strongest performance (mean CTRS score 67.86 +/- 7.24), followed by Qwen 2.5 7b (64.28 +/- 9.55) and Mistral 7b v0.3 (64.17 +/- 9.79), with these differences between models being statistically significant. The CBT-tuned models were competent in implementing core CBT techniques and providing empathetic responses, however, there were limitations observed in agenda adherence, exploration depth and long-context coherence. This study establishes that CBT specific fine-tuning can effectively encode therapeutic competencies in small LLMs, though significant technical and ethical considerations must be resolved prior to clinical deployment.

Related papers

• Test-time Adaptation of Pelvic Bone Segmentation Models via Dynamic Reliability-Guided [62.10276009853225]
  Reliable pelvic bone segmentation from CT is essential for robot-assisted pelvic trauma surgery.<n>Existing methods show limited effectiveness for PBS.<n>We propose a novel closed-loop dynamic Reliability-Guided framework (ReGA) for PBS.
  arXiv  Detail & Related papers  (2026-08-01T08:13:20Z)
• TriALS: Triphasic-Aided Liver Lesion Segmentation Benchmark in Non-Contrast CT [49.93685997940356]
  We describe the TriALS challenge for automated liver lesion segmentation under contrast-limited conditions.<n>The top-performing method achieved a mean venous-phase Dice of 0.754, consistent with human-level performance, yet dropped to 0.57 on NCCT.<n>Algorithm performance was most strongly predicted by training data scale and pre-training strategy.
  arXiv  Detail & Related papers  (2026-05-15T19:23:35Z)
• A Breast Vision Pathology Foundation Model for Real-world Clinical Utility [65.57568187389113]
  We present bfBRAVE, a breast-adaptive pathology foundation model developed and evaluated using a total resource of 101,638 breast wholeslide images.<n>BRAVE supported practical roles in the clinical workflow, including safe exclusion of low-risk cases from routine AI-assisted second-review rescue of initially missed positives and prioritization of cases for further assessment.
  arXiv  Detail & Related papers  (2026-05-06T07:44:39Z)
• TEMPO: Scaling Test-time Training for Large Reasoning Models [87.61789183311856]
  Test-time training (TTT) adapts model parameters on unlabeled test instances during inference time.<n>We propose TEMPO, a TTT framework that interleaves policy refinement on unlabeled questions with periodic critic recalibration on a labeled dataset.
  arXiv  Detail & Related papers  (2026-04-21T10:01:04Z)
• Predicting Neuromodulation Outcome for Parkinson's Disease with Generative Virtual Brain Model [95.41752463487008]
  Parkinson's disease affects over ten million people worldwide.<n>Inter-individual variability limits empirical treatment selection, increasing non-negligible surgical risk and cost.<n>We bridge this gap with a pretraining-finetuning framework to predict outcomes directly from resting-state fMRI.
  arXiv  Detail & Related papers  (2026-03-31T02:36:10Z)
• PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology [48.732366302949515]
  Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.<n>We developed a human-in-the-loop pipeline to create expert rubrics for de-identified patient questions.<n>We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.
  arXiv  Detail & Related papers  (2026-03-02T00:50:39Z)
• AI-Assisted Pleural Effusion Volume Estimation from Contrast-Enhanced CT Images [0.0]
  Pleuralfusions (PE) is a common finding in many different clinical conditions.<n> accurately measuring their volume from CT scans is challenging.<n>We have developed and trained a semi-supervised deep learning framework on contrast-enhanced CT volumes.
  arXiv  Detail & Related papers  (2025-10-04T16:06:10Z)
• The Thinking Therapist: Training Large Language Models to Deliver Acceptance and Commitment Therapy using Supervised Fine-Tuning and Odds Ratio Policy Optimization [0.0]
  Acceptance and Commitment Therapy (ACT) is a cognitive behavioral therapy with emerging evidence of efficacy in several psychiatric conditions.<n>This study investigates the impact of post-training methodology and explicit reasoning on the ability of a small open-weight large language model (LLM) to deliver ACT.
  arXiv  Detail & Related papers  (2025-09-08T02:30:12Z)
• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning [64.36534512742736]
  We investigate the effectiveness of test-time training (TTT) as a mechanism for improving models' reasoning capabilities. TTT significantly improves performance on ARC tasks, achieving up to 6x improvement in accuracy compared to base fine-tuned models. Our findings suggest that explicit symbolic search is not the only path to improved abstract reasoning in neural language models.
  arXiv  Detail & Related papers  (2024-11-11T18:59:45Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy [67.23830698947637]
  We propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios
  arXiv  Detail & Related papers  (2024-10-17T04:52:57Z)
• Dialectical Behavior Therapy Approach to LLM Prompting [1.433758865948252]
  Large language models demonstrated state-of-the-art results on various reasoning tasks when applying the chain-of-thought (CoT) prompting technique. We propose a novel prompting strategy inspired by Dialectical Behavioral Therapy (DBT) We show that prompts crafted with techniques significantly improve results on smaller models.
  arXiv  Detail & Related papers  (2024-10-10T09:58:03Z)
• CACER: Clinical Concept Annotations for Cancer Events and Relations [22.866006682711284]
  We present Clinical Concept s for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48,000 medical problems and drug events. We develop and evaluate transformer-based information extraction models using fine-tuning and in-context learning.
  arXiv  Detail & Related papers  (2024-09-05T20:42:35Z)
• BISeizuRe: BERT-Inspired Seizure Data Representation to Improve Epilepsy Monitoring [13.35453284825286]
  This study presents a novel approach for EEG-based seizure detection leveraging a BERT-based model. The model, BENDR, undergoes a two-phase training process, pre-training and fine-tuning. The optimized model demonstrates substantial performance enhancements, achieving as low as 0.23 FP/h, 2.5$times$ lower than the baseline model, with a lower but still acceptable sensitivity rate.
  arXiv  Detail & Related papers  (2024-06-27T14:09:10Z)
• We Care: Multimodal Depression Detection and Knowledge Infused Mental Health Therapeutic Response Generation [41.09752906121257]
  We present the Extended D-vlog dataset, encompassing a collection of 1, 261 YouTube vlogs. We introduce a virtual agent serving as an initial contact for mental health patients, offering Cognitive Behavioral Therapy (CBT)-based responses. Our Mistral model achieved impressive scores of 70.1% and 30.9% for distortion assessment and classification, along with a Bert score of 88.7%.
  arXiv  Detail & Related papers  (2024-06-15T08:41:46Z)
• Using Pre-training and Interaction Modeling for ancestry-specific disease prediction in UK Biobank [69.90493129893112]
  Recent genome-wide association studies (GWAS) have uncovered the genetic basis of complex traits, but show an under-representation of non-European descent individuals. Here, we assess whether we can improve disease prediction across diverse ancestries using multiomic data.
  arXiv  Detail & Related papers  (2024-04-26T16:39:50Z)
• Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification [52.77024349608834]
  To investigate chest radiograph (CXR) classification performance of vision transformers (ViT) and interpretability of attention-based saliency. ViTs were fine-tuned for lung disease classification using four public data sets: CheXpert, Chest X-Ray 14, MIMIC CXR, and VinBigData. ViTs had comparable CXR classification AUCs compared with state-of-the-art CNNs.
  arXiv  Detail & Related papers  (2023-03-03T12:05:41Z)
• Assessment of Treatment Effect Estimators for Heavy-Tailed Data [70.72363097550483]
  A central obstacle in the objective assessment of treatment effect (TE) estimators in randomized control trials (RCTs) is the lack of ground truth (or validation set) to test their performance. We provide a novel cross-validation-like methodology to address this challenge. We evaluate our methodology across 709 RCTs implemented in the Amazon supply chain.
  arXiv  Detail & Related papers  (2021-12-14T17:53:01Z)
• Multi-institutional Validation of Two-Streamed Deep Learning Method for Automated Delineation of Esophageal Gross Tumor Volume using planning-CT and FDG-PETCT [14.312659667401302]
  Current clinical workflow for esophageal gross tumor volume (GTV) contouring relies on manual delineation of high labor-costs and interuser variability. To validate the clinical applicability of a deep learning (DL) multi-modality esophageal GTV contouring model, developed at 1 institution whereas tested at multiple ones.
  arXiv  Detail & Related papers  (2021-10-11T13:56:09Z)
• Transfer Learning improves MI BCI models classification accuracy in Parkinson's disease patients [50.591267188664666]
  Motor-motorry based BCI (MIBCI) can improve ability and reduce deficit symptoms in Parkinson's Disease patients. Advanced MotorImagery BCI methods are needed to overcome accuracy and time-related calibration challenges. This study proposes a transfer learning-based FBCSP approach which allowed to significantly improve accuracy accuracy in MI BCI performed on PD patients.
  arXiv  Detail & Related papers  (2020-10-29T19:28:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.