MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions

• URL: http://arxiv.org/abs/2212.10720v2
• Date: Fri, 26 May 2023 14:27:14 GMT
• Title: MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions
• Authors: Hao Sun, Zhexin Zhang, Fei Mi, Yasheng Wang, Wei Liu, Jianwei Cui, Bin Wang, Qun Liu, Minlie Huang
• Abstract summary: A moral dialogue system aligned with users' values could enhance conversation engagement and user connections. We propose a framework, MoralDial, to train and evaluate moral dialogue systems.
• Score: 71.25236662907056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Morality in dialogue systems has raised great attention in research recently. A moral dialogue system aligned with users' values could enhance conversation engagement and user connections. In this paper, we propose a framework, MoralDial to train and evaluate moral dialogue systems. In our framework, we first explore the communication mechanisms of morality and resolve expressed morality into three parts, which indicate the roadmap for building a moral dialogue system. Based on that, we design a simple yet effective method: constructing moral discussions between simulated specific users and the dialogue system. The constructed discussions consist of expressing, explaining, revising, and inferring moral views in dialogue exchanges, which makes conversational models learn morality well in a natural manner. Furthermore, we propose a novel evaluation method under the framework. We evaluate the multiple aspects of morality by judging the relation between dialogue responses and human values in discussions, where the multifaceted nature of morality is particularly considered. Automatic and manual experiments demonstrate that our framework is promising to train and evaluate moral dialogue systems.

Related papers

• Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories? [78.3738172874685]
  Making moral judgments is an essential step toward developing ethical AI systems. Prevalent approaches are mostly implemented in a bottom-up manner, which uses a large set of annotated data to train models based on crowd-sourced opinions about morality. This work proposes a flexible top-down framework to steer (Large) Language Models (LMs) to perform moral reasoning with well-established moral theories from interdisciplinary research.
  arXiv  Detail & Related papers  (2023-08-29T15:57:32Z)
• Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversations [14.120888473204907]
  We make and explore connections between moral questions in computer security research and ethics / moral philosophy. We do not seek to define what is morally right or wrong, nor do we argue for one framework over another.
  arXiv  Detail & Related papers  (2023-02-28T05:39:17Z)
• ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations [81.70195684646681]
  We present ClarifyDelphi, an interactive system that learns to ask clarification questions. We posit that questions whose potential answers lead to diverging moral judgments are the most informative. Our work is ultimately inspired by studies in cognitive science that have investigated the flexibility in moral cognition.
  arXiv  Detail & Related papers  (2022-12-20T16:33:09Z)
• A Corpus for Understanding and Generating Moral Stories [84.62366141696901]
  We propose two understanding tasks and two generation tasks to assess these abilities of machines. We present STORAL, a new dataset of Chinese and English human-written moral stories.
  arXiv  Detail & Related papers  (2022-04-20T13:12:36Z)
• The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems [36.90292508433193]
  Moral deviations are difficult to mitigate because moral judgments are not universal. Moral Integrity Corpus captures the moral assumptions of 38k prompt-reply pairs. We show that current neural language models can automatically generate new RoTs that reasonably describe previously unseen interactions.
  arXiv  Detail & Related papers  (2022-04-06T18:10:53Z)
• DynaEval: Unifying Turn and Dialogue Level Evaluation [60.66883575106898]
  We propose DynaEval, a unified automatic evaluation framework. It is capable of performing turn-level evaluation, but also holistically considers the quality of the entire dialogue. Experiments show that DynaEval significantly outperforms the state-of-the-art dialogue coherence model.
  arXiv  Detail & Related papers  (2021-06-02T12:23:18Z)
• Designing for Engaging with News using Moral Framing towards Bridging Ideological Divides [6.177805579183265]
  We present our work designing systems for addressing ideological division through educating U.S. news consumers to engage using a framework of fundamental human values known as Moral Foundations. We design and implement a series of new features that encourage users to challenge their understanding of opposing views. We conduct a field evaluation of each design with 71 participants in total over a period of 6-8 days, finding evidence suggesting users learned to re-frame their discourse in moral values of the opposing side.
  arXiv  Detail & Related papers  (2021-01-27T07:20:48Z)
• Contextualized moral inference [12.574316678945195]
  We present a text-based approach that predicts people's intuitive judgment of moral vignettes. We show that a contextualized representation offers a substantial advantage over alternative representations.
  arXiv  Detail & Related papers  (2020-08-25T00:34:28Z)
• Rethinking Dialogue State Tracking with Reasoning [76.0991910623001]
  This paper proposes to track dialogue states gradually with reasoning over dialogue turns with the help of the back-end data. Empirical results demonstrate that our method significantly outperforms the state-of-the-art methods by 38.6% in terms of joint belief accuracy for MultiWOZ 2.1.
  arXiv  Detail & Related papers  (2020-05-27T02:05:33Z)
• Text-based inference of moral sentiment change [11.188112005462536]
  We present a text-based framework for investigating moral sentiment change of the public via longitudinal corpora. We build our methodology by exploring moral biases learned from diachronic word embeddings. Our work offers opportunities for applying natural language processing toward characterizing moral sentiment change in society.
  arXiv  Detail & Related papers  (2020-01-20T18:52:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.