Decoding moral judgement from text: a pilot study

• URL: http://arxiv.org/abs/2407.00039v1
• Date: Tue, 28 May 2024 20:31:59 GMT
• Title: Decoding moral judgement from text: a pilot study
• Authors: Diana E. Gherman, Thorsten O. Zander,
• Abstract summary: Moral judgement is a complex human reaction that engages cognitive and emotional dimensions. We explore the feasibility of moral judgement decoding from text stimuli with passive brain-computer interfaces.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Moral judgement is a complex human reaction that engages cognitive and emotional dimensions. While some of the morality neural correlates are known, it is currently unclear if we can detect moral violation at a single-trial level. In a pilot study, here we explore the feasibility of moral judgement decoding from text stimuli with passive brain-computer interfaces. For effective moral judgement elicitation, we use video-audio affective priming prior to text stimuli presentation and attribute the text to moral agents. Our results show that further efforts are necessary to achieve reliable classification between moral congruency vs. incongruency states. We obtain good accuracy results for neutral vs. morally-charged trials. With this research, we try to pave the way towards neuroadaptive human-computer interaction and more human-compatible large language models (LLMs)

Related papers

• Morality is Non-Binary: Building a Pluralist Moral Sentence Embedding Space using Contrastive Learning [4.925187725973777]
  Pluralist moral philosophers argue that human morality can be deconstructed into a finite number of elements. We build a pluralist moral sentence embedding space via a state-of-the-art contrastive learning approach. Our results show that a pluralist approach to morality can be captured in an embedding space.
  arXiv  Detail & Related papers  (2024-01-30T18:15:25Z)
• Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories? [78.3738172874685]
  Making moral judgments is an essential step toward developing ethical AI systems. Prevalent approaches are mostly implemented in a bottom-up manner, which uses a large set of annotated data to train models based on crowd-sourced opinions about morality. This work proposes a flexible top-down framework to steer (Large) Language Models (LMs) to perform moral reasoning with well-established moral theories from interdisciplinary research.
  arXiv  Detail & Related papers  (2023-08-29T15:57:32Z)
• MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions [71.25236662907056]
  A moral dialogue system aligned with users' values could enhance conversation engagement and user connections. We propose a framework, MoralDial, to train and evaluate moral dialogue systems.
  arXiv  Detail & Related papers  (2022-12-21T02:21:37Z)
• ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations [81.70195684646681]
  We present ClarifyDelphi, an interactive system that learns to ask clarification questions. We posit that questions whose potential answers lead to diverging moral judgments are the most informative. Our work is ultimately inspired by studies in cognitive science that have investigated the flexibility in moral cognition.
  arXiv  Detail & Related papers  (2022-12-20T16:33:09Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems [36.90292508433193]
  Moral deviations are difficult to mitigate because moral judgments are not universal. Moral Integrity Corpus captures the moral assumptions of 38k prompt-reply pairs. We show that current neural language models can automatically generate new RoTs that reasonably describe previously unseen interactions.
  arXiv  Detail & Related papers  (2022-04-06T18:10:53Z)
• What Would Jiminy Cricket Do? Towards Agents That Behave Morally [59.67116505855223]
  We introduce Jiminy Cricket, an environment suite of 25 text-based adventure games with thousands of diverse, morally salient scenarios. By annotating every possible game state, the Jiminy Cricket environments robustly evaluate whether agents can act morally while maximizing reward. In extensive experiments, we find that the artificial conscience approach can steer agents towards moral behavior without sacrificing performance.
  arXiv  Detail & Related papers  (2021-10-25T17:59:31Z)
• Contextualized moral inference [12.574316678945195]
  We present a text-based approach that predicts people's intuitive judgment of moral vignettes. We show that a contextualized representation offers a substantial advantage over alternative representations.
  arXiv  Detail & Related papers  (2020-08-25T00:34:28Z)
• Aligning AI With Shared Human Values [85.2824609130584]
  We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality. We find that current language models have a promising but incomplete ability to predict basic human ethical judgements. Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.
  arXiv  Detail & Related papers  (2020-08-05T17:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.