Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making

• URL: http://arxiv.org/abs/2605.24319v1
• Date: Sat, 23 May 2026 00:55:36 GMT
• Title: Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making
• Abstract summary: We look for the absence of religious representation as a dimension of value alignment and bias in LLMs.<n>To measure omissive bias, we contribute the AllFaith Religious Representation Benchmark: 150 ethically and personally salient questions.<n>We find that LLMs consistently underrepresent religion relative to human expectations.
• Score: 2.910241872147875
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models become a default source of guidance on personal, moral, and existential questions, it matters whether they draw on the religious frameworks that have historically shaped such reasoning, or systematically omit them. In this paper, we ask a deliberately narrow question: when posed an everyday ethical question for which religious perspectives may be valuable, do LLMs invoke religion at all? In contrast to benchmarks that look for the presence of political leanings or social bias, we look for the absence of religious representation as a dimension of value alignment and bias in LLMs. We term this ``omissive bias.'' To measure omissive bias, we contribute the AllFaith Religious Representation Benchmark: 150 ethically and personally salient questions, sourced from in-the-wild chat transcripts and faith-community contributors, paired with an LLM-as-judge rubric that gives full credit for any mention of a religion, a religious practice, or a religious leader. The questions are not themselves about religion--they are open-ended questions about grief, forgiveness, relationships, purpose, and honesty, where religion is one valuable perspective among several. We also run a human-subjects survey to compare LLM behavior against human expectations. Evaluating 27 models, we find that LLMs consistently underrepresent religion relative to human expectations. The omission is asymmetric: models invoke religion more readily for abstract existential questions (meaning, death, truth) than for the practical personal situations--grief, marriage, family conflict, addiction--where many people most rely on it. It is not our purpose to adjudicate which values LLMs should hold. We argue, more modestly, that current LLM responses overlook critical opportunities to reflect religious frameworks that many people draw on when navigating personal and ethical challenges.

Related papers

• When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance [1.163745353081629]
  When asked for advice on hypothetical faith transitions from one religion to another, models exhibited consistent asymmetries.<n>On average Catholic, Bah', and Sikh religions were broadly favored.<n>Atheists, Agnostics, and Jehovah's Witnesses were primarily disfavored.
  arXiv  Detail & Related papers  (2026-05-21T19:05:09Z)
• Are Language Models Sensitive to Morally Irrelevant Distractors? [47.92026843851412]
  We show that moral distractors can shift the moral judgements of large language models by over 30% even in low-ambiguity scenarios.<n>This research challenges theories that assume the stability of human moral judgements.
  arXiv  Detail & Related papers  (2026-02-10T05:18:05Z)
• Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts [79.1081247754018]
  Large Language Models (LLMs) are widely deployed in reasoning, planning, and decision-making tasks.<n>We propose a framework based on Contact Searching Questions(CSQ) to quantify the likelihood of deception.
  arXiv  Detail & Related papers  (2025-08-08T14:46:35Z)
• Sacred or Synthetic? Evaluating LLM Reliability and Abstention for Religious Questions [10.53116395328794]
  We introduce a novel benchmark FiqhQA focused on the LLM generated Islamic rulings explicitly categorized by the four major Sunni schools of thought, in both Arabic and English.<n>Our zero-shot and abstention experiments reveal significant variation across LLMs, languages, and legal schools of thought.<n>To the best of our knowledge, this is the first study to benchmark the efficacy of LLMs for fine-grained Islamic school of thought specific ruling generation and to evaluate abstention for Islamic queries.
  arXiv  Detail & Related papers  (2025-08-04T07:27:26Z)
• Arbiters of Ambivalence: Challenges of Using LLMs in No-Consensus Tasks [52.098988739649705]
  This study examines the biases and limitations of LLMs in three roles: answer generator, judge, and debater.<n>We develop a no-consensus'' benchmark by curating examples that encompass a variety of a priori ambivalent scenarios.<n>Our results show that while LLMs can provide nuanced assessments when generating open-ended answers, they tend to take a stance on no-consensus topics when employed as judges or debaters.
  arXiv  Detail & Related papers  (2025-05-28T01:31:54Z)
• Normative Evaluation of Large Language Models with Everyday Moral Dilemmas [0.0]
  We evaluate large language models (LLMs) on complex, everyday moral dilemmas sourced from the "Am I the Asshole" (AITA) community on Reddit.<n>Our results demonstrate that large language models exhibit distinct patterns of moral judgment, varying substantially from human evaluations on the AITA subreddit.
  arXiv  Detail & Related papers  (2025-01-30T01:29:46Z)
• Measuring Spiritual Values and Bias of Large Language Models [28.892254056685008]
  Large language models (LLMs) have become integral tool for users from various backgrounds.<n>These models reflect linguistic and cultural nuances embedded in pre-training data.<n> values and perspectives inherent in this data can influence the behavior of LLMs, leading to potential biases.
  arXiv  Detail & Related papers  (2024-10-15T14:33:23Z)
• Divine LLaMAs: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Models [19.54202714712677]
  Religion as a socio-cultural system prescribes a set of beliefs and values for its followers. Unlike gender, which says little about our values, religion prescribes a set of beliefs and values for its followers. Major religions in the US and European countries are represented with more nuance. Eastern religions like Hinduism and Buddhism are strongly stereotyped.
  arXiv  Detail & Related papers  (2024-07-09T14:45:15Z)
• Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
  We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
  arXiv  Detail & Related papers  (2024-03-15T04:02:24Z)
• Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models [61.45529177682614]
  We challenge the prevailing constrained evaluation paradigm for values and opinions in large language models. We show that models give substantively different answers when not forced. We distill these findings into recommendations and open challenges in evaluating values and opinions in LLMs.
  arXiv  Detail & Related papers  (2024-02-26T18:00:49Z)
• Large Language Models are Geographically Biased [47.88767211956144]
  We study what Large Language Models (LLMs) know about the world we live in through the lens of geography. We show various problematic geographic biases, which we define as systemic errors in geospatial predictions.
  arXiv  Detail & Related papers  (2024-02-05T02:32:09Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks [0.0]
  We consider the questions of whether or not large language models (LLMs) have beliefs, and, if they do, how we might measure them. We evaluate two existing approaches, one due to Azaria and Mitchell (2023) and the other to Burns et al. (2022). We argue that, even if LLMs have beliefs, these methods are unlikely to be successful for conceptual reasons. Thus, there is still no lie-detector for LLMs.
  arXiv  Detail & Related papers  (2023-06-30T23:44:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.