DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generation

• URL: http://arxiv.org/abs/2306.01657v2
• Date: Sun, 9 Jul 2023 08:09:49 GMT
• Title: DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generation
• Abstract summary: We propose to use explicit control to guide the empathy expression and design a framework DiffusEmp based on conditional diffusion language model. We show that our framework outperforms competitive baselines in terms of controllability, informativeness, and diversity without the loss of context-relatedness.
• Score: 26.73326709474638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Empathy is a crucial factor in open-domain conversations, which naturally shows one's caring and understanding to others. Though several methods have been proposed to generate empathetic responses, existing works often lead to monotonous empathy that refers to generic and safe expressions. In this paper, we propose to use explicit control to guide the empathy expression and design a framework DiffusEmp based on conditional diffusion language model to unify the utilization of dialogue context and attribute-oriented control signals. Specifically, communication mechanism, intent, and semantic frame are imported as multi-grained signals that control the empathy realization from coarse to fine levels. We then design a specific masking strategy to reflect the relationship between multi-grained signals and response tokens, and integrate it into the diffusion model to influence the generative process. Experimental results on a benchmark dataset EmpatheticDialogue show that our framework outperforms competitive baselines in terms of controllability, informativeness, and diversity without the loss of context-relatedness.

Related papers

• Emotion-Cause Pair Extraction in Conversations via Semantic Decoupling and Graph Alignment [8.345559401830291]
  ECPEC aims to identify the set of causal relations between emotion utterances and their triggering causes within a dialogue.<n>We seek to disentangle emotion-oriented semantics from cause-oriented semantics, mapping them into two complementary representation spaces.<n>We propose a unified framework that instantiates the above semantic decoupling and alignment principle within a shared conversational structure.
  arXiv  Detail & Related papers  (2026-04-21T15:05:48Z)
• Dual-branch Graph Domain Adaptation for Cross-scenario Multi-modal Emotion Recognition [43.17664347937779]
  Multimodal Emotion Recognition in Conversations (MERC) aims to predict speakers' emotional states in multi-turn dialogues through text, audio, and visual cues.<n>Existing MERC methods generally neglect cross-scenario variations, limiting their ability to transfer models trained on a source domain to unseen target domains.<n>We propose a Dual-branch Graph Adaptation framework (DGDA) for multimodal emotion recognition under cross-scenario conditions.
  arXiv  Detail & Related papers  (2026-03-27T08:21:09Z)
• Geospatial-Reasoning-Driven Vocabulary-Agnostic Remote Sensing Semantic Segmentation [13.743073097114461]
  Open-vocabulary semantic segmentation has emerged as a promising research direction in remote sensing.<n>We propose a Geospatial Reasoning Chain-of-Thought (GR-CoT) framework to guide open-vocabulary segmentation models toward precise mapping.
  arXiv  Detail & Related papers  (2026-02-09T02:09:21Z)
• Explaining multimodal LLMs via intra-modal token interactions [55.27436637894534]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse vision-language tasks, yet their internal decision-making mechanisms remain insufficiently understood.<n>We propose enhancing interpretability by leveraging intra-modal interaction.
  arXiv  Detail & Related papers  (2025-09-26T14:39:13Z)
• ControlThinker: Unveiling Latent Semantics for Controllable Image Generation through Visual Reasoning [76.2503352325492]
  ControlThinker is a novel framework that employs a "comprehend-then-generate" paradigm.<n>Latent semantics from control images are mined to enrich text prompts.<n>This enriched semantic understanding then seamlessly aids in image generation without the need for additional complex modifications.
  arXiv  Detail & Related papers  (2025-06-04T05:56:19Z)
• Cross-modal Context Fusion and Adaptive Graph Convolutional Network for Multimodal Conversational Emotion Recognition [0.0]
  This article proposes a new multimodal emotion recognition method, including a cross modal context fusion module, an adaptive graph convolutional encoding module, and an emotion classification module. Our model has surpassed some state-of-the-art methods on publicly available benchmark datasets and achieved high recognition accuracy.
  arXiv  Detail & Related papers  (2025-01-25T03:53:53Z)
• Effective Context Modeling Framework for Emotion Recognition in Conversations [2.7175580940471913]
  Emotion Recognition in Conversations (ERC) facilitates a deeper understanding of the emotions conveyed by speakers in each utterance within a conversation. Recent Graph Neural Networks (GNNs) have demonstrated their strengths in capturing data relationships. We propose ConxGNN, a novel GNN-based framework designed to capture contextual information in conversations.
  arXiv  Detail & Related papers  (2024-12-21T02:22:06Z)
• PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis [74.41260927676747]
  This paper bridges the gaps by introducing a multimodal conversational Sentiment Analysis (ABSA) To benchmark the tasks, we construct PanoSent, a dataset annotated both manually and automatically, featuring high quality, large scale, multimodality, multilingualism, multi-scenarios, and covering both implicit and explicit sentiment elements. To effectively address the tasks, we devise a novel Chain-of-Sentiment reasoning framework, together with a novel multimodal large language model (namely Sentica) and a paraphrase-based verification mechanism.
  arXiv  Detail & Related papers  (2024-08-18T13:51:01Z)
• AMuSE: Adaptive Multimodal Analysis for Speaker Emotion Recognition in Group Conversations [39.79734528362605]
  Multimodal Attention Network captures cross-modal interactions at various levels of spatial abstraction. AMuSE model condenses both spatial and temporal features into two dense descriptors: speaker-level and utterance-level.
  arXiv  Detail & Related papers  (2024-01-26T19:17:05Z)
• A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations [15.77747948751497]
  We propose a transformer-based model with self-distillation (SDT) for the task. The proposed model captures intra- and inter-modal interactions by utilizing intra- and inter-modal transformers. We introduce self-distillation to transfer knowledge of hard and soft labels from the proposed model to each modality.
  arXiv  Detail & Related papers  (2023-10-31T14:33:30Z)
• A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition [72.36055502078193]
  We propose a hierarchical framework, based on chain regression models, for affective recognition from vocal bursts. To address the challenge of data sparsity, we also use self-supervised learning (SSL) representations with layer-wise and temporal aggregation modules. The proposed systems participated in the ACII Affective Vocal Burst (A-VB) Challenge 2022 and ranked first in the "TWO'' and "CULTURE" tasks.
  arXiv  Detail & Related papers  (2023-03-14T16:08:45Z)
• Bridging the Emotional Semantic Gap via Multimodal Relevance Estimation [9.632084332065523]
  We propose a multimodal relevance estimation network to capture the relevant semantics among modalities in multimodal emotions. We make use of contrastive learning to optimize the similarity of category-level modality-relevant semantics across different modalities in feature space. In order to better reflect the emotional state in the real interactive scenarios, we collect a single-label discrete multimodal emotion dataset named SDME.
  arXiv  Detail & Related papers  (2023-02-03T05:27:52Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation [59.01297461453444]
  We propose a hierarchical contrastive learning mechanism, which can unify hybrid granularities semantic meaning in the input text. Experiments demonstrate that our model outperforms competitive baselines on paraphrasing, dialogue generation, and storytelling tasks.
  arXiv  Detail & Related papers  (2022-05-26T13:26:03Z)
• Semantic-Enhanced Explainable Finetuning for Open-Domain Dialogues [33.50099424582726]
  We propose to combine pretrained language models with the modular dialogue paradigm for open-domain dialogue modeling. Our method, semantic-enhanced finetuning, instantiates conversation understanding, planning, and response generation as a language model finetuning task.
  arXiv  Detail & Related papers  (2021-06-06T09:03:41Z)
• CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation [41.28609999443215]
  We propose a multi-factor hierarchical framework, CoMAE, for empathetic response generation. We show experimentally that our CoMAE-based model can generate more empathetic responses than previous methods.
  arXiv  Detail & Related papers  (2021-05-18T07:13:33Z)
• Linguistic Structure Guided Context Modeling for Referring Image Segmentation [61.701577239317785]
  We propose a "gather-propagate-distribute" scheme to model multimodal context by cross-modal interaction. Our LSCM module builds a Dependency Parsing Tree Word Graph (DPT-WG) which guides all the words to include valid multimodal context of the sentence.
  arXiv  Detail & Related papers  (2020-10-01T16:03:51Z)
• A Controllable Model of Grounded Response Generation [122.7121624884747]
  Current end-to-end neural conversation models inherently lack the flexibility to impose semantic control in the response generation process. We propose a framework that we call controllable grounded response generation (CGRG) We show that using this framework, a transformer based model with a novel inductive attention mechanism, trained on a conversation-like Reddit dataset, outperforms strong generation baselines.
  arXiv  Detail & Related papers  (2020-05-01T21:22:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.