Steering Conversational Large Language Models for Long Emotional Support
Conversations
- URL: http://arxiv.org/abs/2402.10453v1
- Date: Fri, 16 Feb 2024 05:03:01 GMT
- Title: Steering Conversational Large Language Models for Long Emotional Support
Conversations
- Authors: Navid Madani, Sougata Saha, Rohini Srihari
- Abstract summary: We introduce the Strategy-Relevant Attention metric, a model-agnostic measure designed to evaluate the effectiveness of large language models (LLMs) in adhering to strategic prompts in emotional support contexts.
Our findings reveal that the application of SRA-informed prompts leads to enhanced strategic adherence, resulting in conversations that more reliably exhibit the desired emotional support strategies over longer conversations.
- Score: 5.601537787608725
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this study, we address the challenge of consistently following emotional
support strategies in long conversations by large language models (LLMs). We
introduce the Strategy-Relevant Attention (SRA) metric, a model-agnostic
measure designed to evaluate the effectiveness of LLMs in adhering to strategic
prompts in emotional support contexts. By analyzing conversations within the
Emotional Support Conversations dataset (ESConv) using LLaMA models, we
demonstrate that SRA is significantly correlated with a model's ability to
sustain the outlined strategy throughout the interactions. Our findings reveal
that the application of SRA-informed prompts leads to enhanced strategic
adherence, resulting in conversations that more reliably exhibit the desired
emotional support strategies over longer conversations. Furthermore, we
contribute a comprehensive, multi-branch synthetic conversation dataset for
ESConv, featuring a variety of strategy continuations informed by our optimized
prompting method. The code and data are publicly available on our Github.
Related papers
- LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models [75.89014602596673]
Strategic reasoning requires understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly.
We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with Large Language Models.
It underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.
arXiv Detail & Related papers (2024-04-01T16:50:54Z) - Towards Understanding Counseling Conversations: Domain Knowledge and
Large Language Models [22.588557390720236]
This paper proposes a systematic approach to examine the efficacy of domain knowledge and large language models (LLMs) in better representing counseling conversations.
We empirically show that state-of-the-art language models such as Transformer-based models and GPT models fail to predict the conversation outcome.
arXiv Detail & Related papers (2024-02-22T01:02:37Z) - Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation [28.74445806009475]
This work initially analyzes the results of large language models (LLMs) on ESConv.
We observe that exhibiting high preference for specific strategies hinders effective emotional support.
Our findings emphasize that (1) low preference for specific strategies hinders the progress of emotional support, (2) external assistance helps reduce preference bias, and (3) existing LLMs alone cannot become good emotional supporters.
arXiv Detail & Related papers (2024-02-20T18:21:32Z) - Self-Explanation Prompting Improves Dialogue Understanding in Large
Language Models [52.24756457516834]
We propose a novel "Self-Explanation" prompting strategy to enhance the comprehension abilities of Large Language Models (LLMs)
This task-agnostic approach requires the model to analyze each dialogue utterance before task execution, thereby improving performance across various dialogue-centric tasks.
Experimental results from six benchmark datasets confirm that our method consistently outperforms other zero-shot prompts and matches or exceeds the efficacy of few-shot prompts.
arXiv Detail & Related papers (2023-09-22T15:41:34Z) - Building Emotional Support Chatbots in the Era of LLMs [64.06811786616471]
We introduce an innovative methodology that synthesizes human insights with the computational prowess of Large Language Models (LLMs)
By utilizing the in-context learning potential of ChatGPT, we generate an ExTensible Emotional Support dialogue dataset, named ExTES.
Following this, we deploy advanced tuning techniques on the LLaMA model, examining the impact of diverse training strategies, ultimately yielding an LLM meticulously optimized for emotional support interactions.
arXiv Detail & Related papers (2023-08-17T10:49:18Z) - Prompting and Evaluating Large Language Models for Proactive Dialogues:
Clarification, Target-guided, and Non-collaboration [72.04629217161656]
This work focuses on three aspects of proactive dialogue systems: clarification, target-guided, and non-collaborative dialogues.
To trigger the proactivity of LLMs, we propose the Proactive Chain-of-Thought prompting scheme.
arXiv Detail & Related papers (2023-05-23T02:49:35Z) - PoKE: Prior Knowledge Enhanced Emotional Support Conversation with
Latent Variable [1.5787128553734504]
The emotional support is a critical communication skill that should be trained into dialogue systems.
Most existing studies predict support strategy according to current context and provide corresponding emotional support in response.
We propose Prior Knowledge Enhanced emotional support conversation with latent variable model, PoKE.
arXiv Detail & Related papers (2022-10-23T07:31:24Z) - Improving Multi-turn Emotional Support Dialogue Generation with
Lookahead Strategy Planning [81.79431311952656]
We propose a novel system MultiESC to provide Emotional Support.
For strategy planning, we propose lookaheads to estimate the future user feedback after using particular strategies.
For user state modeling, MultiESC focuses on capturing users' subtle emotional expressions and understanding their emotion causes.
arXiv Detail & Related papers (2022-10-09T12:23:47Z) - RESPER: Computationally Modelling Resisting Strategies in Persuasive
Conversations [0.7505101297221454]
We propose a generalised framework for identifying resisting strategies in persuasive conversations.
Our experiments reveal the asymmetry of power roles in non-collaborative goal-directed conversations.
We also investigate the role of different resisting strategies on the conversation outcome.
arXiv Detail & Related papers (2021-01-26T03:44:17Z) - Dialogue-Based Relation Extraction [53.2896545819799]
We present the first human-annotated dialogue-based relation extraction (RE) dataset DialogRE.
We argue that speaker-related information plays a critical role in the proposed task, based on an analysis of similarities and differences between dialogue-based and traditional RE tasks.
Experimental results demonstrate that a speaker-aware extension on the best-performing model leads to gains in both the standard and conversational evaluation settings.
arXiv Detail & Related papers (2020-04-17T03:51:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.