Navigating through the hidden embedding space: steering LLMs to improve mental health assessment

• URL: http://arxiv.org/abs/2510.16373v1
• Date: Sat, 18 Oct 2025 06:51:39 GMT
• Title: Navigating through the hidden embedding space: steering LLMs to improve mental health assessment
• Abstract summary: Small-scale AI models still struggle to deliver optimal performance in domain-specific applications.<n>Our lightweight method consists of a linear transformation applied to a specific layer's activations, leveraging steering vectors to guide the model's output.
• Score: 5.788118988775462
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid evolution of Large Language Models (LLMs) is transforming AI, opening new opportunities in sensitive and high-impact areas such as Mental Health (MH). Yet, despite these advancements, recent evidence reveals that smaller-scale models still struggle to deliver optimal performance in domain-specific applications. In this study, we present a cost-efficient yet powerful approach to improve MH assessment capabilities of an LLM, without relying on any computationally intensive techniques. Our lightweight method consists of a linear transformation applied to a specific layer's activations, leveraging steering vectors to guide the model's output. Remarkably, this intervention enables the model to achieve improved results across two distinct tasks: (1) identifying whether a Reddit post is useful for detecting the presence or absence of depressive symptoms (relevance prediction task), and (2) completing a standardized psychological screening questionnaire for depression based on users' Reddit post history (questionnaire completion task). Results highlight the untapped potential of steering mechanisms as computationally efficient tools for LLMs' MH domain adaptation.

Related papers

• FormuEvo: LLM-Guided Evolution for Discovering Solver-Efficient Mixed-Integer Programming Formulations [66.32852384262038]
  Mixed-integer programming (MIP) lies at the core of operations research and industrial optimization.<n>We propose FormuEvo, an evolutionary framework for automated discovery of solver-efficient MIP formulations.<n>FormuEvo frames MIP formulation design as evolutionary optimization over the symbolic space of MIP formulations.
  arXiv  Detail & Related papers  (2026-08-24T15:01:28Z)
• PyVRP$^+$: LLM-Driven Metacognitive Heuristic Evolution for Hybrid Genetic Search in Vehicle Routing Problems [30.443372626623887]
  We introduce Metacognitive Programming (MEP), a framework that elevates a large language model to a strategic discovery agent.<n>Our results show that MEP discoverss that yield significant performance gains over the original HGS baseline, improving solution quality by up to 2.70% and reducing runtime by over 45% on challenging VRP variants.
  arXiv  Detail & Related papers  (2026-04-09T06:36:15Z)
• SAPO: Self-Adaptive Process Optimization Makes Small Reasoners Stronger [13.044308337439249]
  Existing self-evolution methods overlook the influence of fine-grained reasoning steps, which leads to the reasoner-verifier gap.<n>Motivated by the Error-Related Negativity (ERN), we propose a Self-Adaptive Process Optimization (SAPO) method for self-improvement in Small Language Models (SLMs)
  arXiv  Detail & Related papers  (2026-01-28T07:04:30Z)
• LMILAtt: A Deep Learning Model for Depression Detection from Social Media Users Enhanced by Multi-Instance Learning Based on Attention Mechanism [2.398386906858336]
  Depression is a major global public health challenge and its early identification is crucial.<n>This study proposes the LMILAtt model, which integrates Long Short-Term Memory autoencoders and attention mechanisms.<n>Experiments show that the model is significantly better than the baseline model in terms of accuracy, recall and F1 score.
  arXiv  Detail & Related papers  (2025-09-30T11:58:32Z)
• Agentic Reinforced Policy Optimization [66.96989268893932]
  Large-scale reinforcement learning with verifiable rewards (RLVR) has demonstrated its effectiveness in harnessing the potential of large language models (LLMs) for single-turn reasoning tasks.<n>Current RL algorithms inadequately balance the models' intrinsic long-horizon reasoning capabilities and their proficiency in multi-turn tool interactions.<n>We propose Agentic Reinforced Policy Optimization (ARPO), a novel agentic RL algorithm tailored for training multi-turn LLM-based agents.
  arXiv  Detail & Related papers  (2025-07-26T07:53:11Z)
• Hierarchical Reinforcement Learning Framework for Adaptive Walking Control Using General Value Functions of Lower-Limb Sensor Signals [0.40498500266986387]
  We explore the use of Hierarchical Reinforcement Learning to develop adaptive control strategies for lower-limb exoskeletons.<n>Our approach breaks down the complex task of exoskeleton control adaptation into a higher-level framework for terrain strategy adaptation and a lower-level framework for providing predictive information.<n>We investigated two methods for incorporating actual and predicted sensor signals into a policy network with the intent to improve the decision-making capacity of the control system.
  arXiv  Detail & Related papers  (2025-07-22T19:47:04Z)
• Exploring Model Editing for LLM-based Aspect-Based Sentiment Classification [17.512415475301395]
  We investigate model editing to serve an efficient method for adapting large language models (LLMs) to solve aspect-based sentiment classification.<n>Our findings reveal that a distinct set of mid-layer representations is essential for detecting the sentiment polarity of given aspect words.<n>We develop a model editing approach that focuses exclusively on these critical parts of the LLM, leading to a more efficient method for adapting LLMs.
  arXiv  Detail & Related papers  (2025-03-19T11:21:37Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions.<n>We propose a novel approach utilizing structured medical reasoning.<n>Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search [57.28671084993782]
  Large language models (LLMs) have demonstrated remarkable reasoning capabilities across diverse domains.<n>Recent studies have shown that increasing test-time computation enhances LLMs' reasoning capabilities.<n>We propose a two-stage training paradigm: 1) a small-scale format tuning stage to internalize the COAT reasoning format and 2) a large-scale self-improvement stage leveraging reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-04T17:26:58Z)
• Adaptive Pruning for Large Language Models with Structural Importance Awareness [66.2690963378878]
  Large language models (LLMs) have significantly improved language understanding and generation capabilities.<n>LLMs are difficult to deploy on resource-constrained edge devices due to their high computational and storage resource demands.<n>We propose structurally-aware adaptive pruning (SAAP) to significantly reduce the computational and memory costs while maintaining model performance.
  arXiv  Detail & Related papers  (2024-12-19T18:08:04Z)
• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models [37.476241509187304]
  Large Language Models (LLMs) achieve remarkable performance through pretraining on extensive data.<n>The lack of interpretability in their underlying mechanisms limits the ability to effectively steer LLMs for specific applications.<n>We introduce an efficient selective layer intervention based on prominent parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-10-23T09:40:15Z)
• X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection [55.77552681618732]
  X2-DFD is an eXplainable and eXtendable framework based on multimodal large-language models (MLLMs) for deepfake detection.<n>The first stage, Model Feature Assessment, systematically evaluates the detectability of forgery-related features for the MLLM.<n>The second stage, Explainable dataset Construction, consists of two key modules: Strong Feature Strengthening and Weak Feature Supplementing.<n>The third stage, Fine-tuning and Inference, involves fine-tuning the MLLM on the constructed dataset and deploying it for final detection and explanation.
  arXiv  Detail & Related papers  (2024-10-08T15:28:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.