Language Models have a Moral Dimension

• URL: http://arxiv.org/abs/2103.11790
• Date: Mon, 8 Mar 2021 16:59:52 GMT
• Title: Language Models have a Moral Dimension
• Authors: Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin Rothkopf, Kristian Kersting
• Abstract summary: We show that recent improvements of LMs store ethical and moral values of the society. This provides a path for attenuating or even preventing toxic degeneration in LMs.
• Score: 13.991481870016631
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial writing is permeating our lives due to recent advances in large-scale, transformer-based language models (LMs) such as BERT, its variants, GPT-2/3, and others. Using them as pretrained models and fine-tuning them for specific tasks, researchers have extended the state of the art for many NLP tasks and shown that they not only capture linguistic knowledge but also retain general knowledge implicitly present in the data. These and other successes are exciting. Unfortunately, LMs trained on unfiltered text corpora suffer from degenerate and biased behaviour. While this is well established, we show that recent improvements of LMs also store ethical and moral values of the society and actually bring a ``moral dimension'' to surface: the values are capture geometrically by a direction in the embedding space, reflecting well the agreement of phrases to social norms implicitly expressed in the training texts. This provides a path for attenuating or even preventing toxic degeneration in LMs. Since one can now rate the (non-)normativity of arbitrary phrases without explicitly training the LM for this task, the moral dimension can be used as ``moral compass'' guiding (even other) LMs towards producing normative text, as we will show.

Related papers

• SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation [55.61004653386632]
  Large Language Models (LLMs) often produce hallucinations, i.e., information that is unfaithful or not grounded in the input context. This paper introduces a novel self-supervised method for generating a training set of unfaithful samples. We then refine the model using a training process that encourages the generation of grounded outputs over unfaithful ones.
  arXiv  Detail & Related papers  (2025-02-19T12:31:58Z)
• How Inclusively do LMs Perceive Social and Moral Norms? [5.302888878095751]
  Language models (LMs) are used in decision-making systems and as interactive assistants. We investigate how inclusively LMs perceive norms across demographic groups. We find notable disparities in LM responses, with younger, higher-income groups showing closer alignment.
  arXiv  Detail & Related papers  (2025-02-04T20:24:17Z)
• Smaller Large Language Models Can Do Moral Self-Correction [7.899707459486236]
  Self-correction is one of the most amazing emerging capabilities of Large Language Models (LLMs) Moral self-correction is a post-hoc approach correcting unethical generations without requiring a gradient update. Previous works have shown that LLMs can self-debias, and it has been reported that small models, i.e., those with less than 22B parameters, are not capable of moral self-correction.
  arXiv  Detail & Related papers  (2024-10-30T22:58:57Z)
• Measuring Moral Inconsistencies in Large Language Models [16.47371312298185]
  A Large Language Model (LLM) is considered consistent if semantically equivalent prompts produce semantically equivalent responses. We show that even state-of-the-art LLMs are highly inconsistent in their generations, questioning their reliability. We propose a novel information-theoretic measure called Semantic Graph Entropy (SGE) to measure the consistency of an LLM in moral scenarios.
  arXiv  Detail & Related papers  (2024-01-26T18:05:47Z)
• Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learning [36.66806788879868]
  Large Language Models (LLMs) have made unprecedented breakthroughs, yet their integration into everyday life might raise societal risks due to generated unethical content. This work delves into ethical values utilizing Moral Foundation Theory.
  arXiv  Detail & Related papers  (2023-10-17T07:42:40Z)
• "According to ...": Prompting Language Models Improves Quoting from Pre-Training Data [52.03853726206584]
  Large Language Models (LLMs) may hallucinate and generate fake information, despite pre-training on factual data. We propose according-to prompting: directing LLMs to ground responses against previously observed text. To quantify this grounding, we propose a novel evaluation metric (QUIP-Score) that measures the extent to which model-produced answers are directly found in underlying text corpora.
  arXiv  Detail & Related papers  (2023-05-22T17:25:24Z)
• Augmented Language Models: a Survey [55.965967655575454]
  This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. We refer to them as Augmented Language Models (ALMs) The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks.
  arXiv  Detail & Related papers  (2023-02-15T18:25:52Z)
• Language Models of Code are Few-Shot Commonsense Learners [106.1531522893209]
  Given a natural language input, the goal is to generate a graph such as an event -- or a reasoning-graph. Existing approaches serialize the output graph as a flat list of nodes and edges. We show that when we instead frame structured commonsense reasoning tasks as code generation tasks, pre-trained LMs of code are better structured commonsense reasoners than LMs of natural language.
  arXiv  Detail & Related papers  (2022-10-13T16:09:36Z)
• Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity [0.0]
  This study investigates whether Large Language Models reproduce the moral biases associated with political groups in the United States. Using tools from Moral Foundations Theory, it is shown that these LLMs are indeed moral mimics.
  arXiv  Detail & Related papers  (2022-09-24T23:55:53Z)
• Factuality Enhanced Language Models for Open-Ended Text Generation [60.27166549575472]
  We design the FactualityPrompts test set and metrics to measure the factuality of LM generations. We find that larger LMs are more factual than smaller ones, although a previous study suggests that larger LMs can be less truthful in terms of misconceptions. We propose a factuality-enhanced training method that uses TopicPrefix for better awareness of facts and sentence completion.
  arXiv  Detail & Related papers  (2022-06-09T17:16:43Z)
• Towards Language Modelling in the Speech Domain Using Sub-word Linguistic Units [56.52704348773307]
  We propose a novel LSTM-based generative speech LM based on linguistic units including syllables and phonemes. With a limited dataset, orders of magnitude smaller than that required by contemporary generative models, our model closely approximates babbling speech. We show the effect of training with auxiliary text LMs, multitask learning objectives, and auxiliary articulatory features.
  arXiv  Detail & Related papers  (2021-10-31T22:48:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.