Mitigating Toxic Degeneration with Empathetic Data: Exploring the Relationship Between Toxicity and Empathy

• URL: http://arxiv.org/abs/2205.07233v1
• Date: Sun, 15 May 2022 09:37:15 GMT
• Title: Mitigating Toxic Degeneration with Empathetic Data: Exploring the Relationship Between Toxicity and Empathy
• Authors: Allison Lahnala, Charles Welch, B\'ela Neuendorf, Lucie Flek
• Abstract summary: Using empathetic data, we improve over recent work on controllable text generation that aims to reduce the toxicity of generated text. We find we are able to dramatically reduce the size of fine-tuning data to 7.5-30k samples while at the same time making significant improvements over state-of-the-art toxicity mitigation.
• Score: 8.293498506807333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large pre-trained neural language models have supported the effectiveness of many NLP tasks, yet are still prone to generating toxic language hindering the safety of their use. Using empathetic data, we improve over recent work on controllable text generation that aims to reduce the toxicity of generated text. We find we are able to dramatically reduce the size of fine-tuning data to 7.5-30k samples while at the same time making significant improvements over state-of-the-art toxicity mitigation of up to 3.4% absolute reduction (26% relative) from the original work on 2.3m samples, by strategically sampling data based on empathy scores. We observe that the degree of improvement is subject to specific communication components of empathy. In particular, the cognitive components of empathy significantly beat the original dataset in almost all experiments, while emotional empathy was tied to less improvement and even underperforming random samples of the original data. This is a particularly implicative insight for NLP work concerning empathy as until recently the research and resources built for it have exclusively considered empathy as an emotional concept.

Related papers

• AI chatbots versus human healthcare professionals: a systematic review and meta-analysis of empathy in patient care [0.10262304700896197]
  Use of artificial intelligence (AI)-based chatbots in healthcare is rapidly expanding.<n>Some studies suggest AI chatbots can outperform human healthcare professionals (HCPs) in empathy.<n>We searched multiple databases for studies comparing AI chatbots with human HCPs on empathy measures.<n>Thirteen studies reported statistically significantly higher empathy ratings for AI, with only two studies situated in dermatology favouring human responses.
  arXiv  Detail & Related papers  (2026-02-05T13:09:19Z)
• <think> So let's replace this phrase with insult... </think> Lessons learned from generation of toxic texts with LLMs [60.169913160819]
  This paper explores the possibility of using synthetic toxic data as an alternative to human-generated data for training models for detoxification.<n>Experiments show that models fine-tuned on synthetic data consistently perform worse than those trained on human data.<n>The root cause is identified as a critical lexical diversity gap: LLMs generate toxic content using a small, repetitive vocabulary of insults that fails to capture the nuances and variety of human toxicity.
  arXiv  Detail & Related papers  (2025-09-10T07:48:24Z)
• Analyzing Toxicity in Open Source Software Communications Using Psycholinguistics and Moral Foundations Theory [5.03553492616371]
  This paper investigates a machine learning-based approach for the automatic detection of toxic communications in Open Source Software (OSS) We leverage psycholinguistic lexicons, and Moral Foundations Theory to analyze toxicity in two types of OSS communication channels; issue comments and code reviews. Using moral values as features is more effective than linguistic cues, resulting in 67.50% F1-measure in identifying toxic instances in code review data and 64.83% in issue comments.
  arXiv  Detail & Related papers  (2024-12-17T17:52:00Z)
• APTNESS: Incorporating Appraisal Theory and Emotion Support Strategies for Empathetic Response Generation [71.26755736617478]
  Empathetic response generation is designed to comprehend the emotions of others. We develop a framework that combines retrieval augmentation and emotional support strategy integration. Our framework can enhance the empathy ability of LLMs from both cognitive and affective empathy perspectives.
  arXiv  Detail & Related papers  (2024-07-23T02:23:37Z)
• Efficient-Empathy: Towards Efficient and Effective Selection of Empathy Data [32.483540066357]
  We present Efficient-Empathy, a sensibility and rationality score-based data selection algorithm. Our trained sensibility model achieves efficiently state-of-the-art (SoTA) performance. By integrating sensibility and rationality data with a MoE structure, we achieve even higher performance.
  arXiv  Detail & Related papers  (2024-07-02T04:11:52Z)
• Empathy and Distress Detection using Ensembles of Transformer Models [0.0]
  This paper presents our approach for the WASSA 2023 Empathy, Emotion and Personality Shared Task. Empathy and distress detection are crucial challenges in Natural Language Processing. Our final submission has a Pearson's r score of 0.346, placing us third in the empathy and distress detection subtask.
  arXiv  Detail & Related papers  (2023-12-05T08:50:34Z)
• Relevance in Dialogue: Is Less More? An Empirical Comparison of Existing Metrics, and a Novel Simple Metric [18.690461703947047]
  We propose modifications to reduce data requirements and domain sensitivity while improving correlation. Our proposed metric achieves state-of-the-art performance on the HUMOD dataset while reducing measured sensitivity to dataset by 37%-66%.
  arXiv  Detail & Related papers  (2022-06-03T21:23:05Z)
• Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication [88.52901763928045]
  We propose an approach that relies on exemplars to cue the generative model on fine stylistic properties that signal empathy to the interlocutor. We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
  arXiv  Detail & Related papers  (2021-06-22T14:02:33Z)
• Mitigating Biases in Toxic Language Detection through Invariant Rationalization [70.36701068616367]
  biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. We propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns. Our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.
  arXiv  Detail & Related papers  (2021-06-14T08:49:52Z)
• Text Mining to Identify and Extract Novel Disease Treatments From Unstructured Datasets [56.38623317907416]
  We use Google Cloud to transcribe podcast episodes of an NPR radio show. We then build a pipeline for systematically pre-processing the text. Our model successfully identified that Omeprazole can help treat heartburn.
  arXiv  Detail & Related papers  (2020-10-22T19:52:49Z)
• RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models [93.151822563361]
  Pretrained neural language models (LMs) are prone to generating racist, sexist, or otherwise toxic language which hinders their safe deployment. We investigate the extent to which pretrained LMs can be prompted to generate toxic language, and the effectiveness of controllable text generation algorithms at preventing such toxic degeneration.
  arXiv  Detail & Related papers  (2020-09-24T03:17:19Z)
• Towards Persona-Based Empathetic Conversational Models [58.65492299237112]
  Empathetic conversational models have been shown to improve user satisfaction and task outcomes in numerous domains. In Psychology, persona has been shown to be highly correlated to personality, which in turn influences empathy. We propose a new task towards persona-based empathetic conversations and present the first empirical study on the impact of persona on empathetic responding.
  arXiv  Detail & Related papers  (2020-04-26T08:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.