UbuntuGuard: A Culturally-Grounded Policy Benchmark for Equitable AI Safety in African Languages

• URL: http://arxiv.org/abs/2601.12696v1
• Date: Mon, 19 Jan 2026 03:37:56 GMT
• Title: UbuntuGuard: A Culturally-Grounded Policy Benchmark for Equitable AI Safety in African Languages
• Abstract summary: Current guardian models are predominantly Western-centric and optimized for high-resource languages.<n>We introduce UbuntuGuard, the first African policy-based safety benchmark built from adversarial queries authored by 155 domain experts.
• Score: 18.40701733030824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current guardian models are predominantly Western-centric and optimized for high-resource languages, leaving low-resource African languages vulnerable to evolving harms, cross-lingual safety failures, and cultural misalignment. Moreover, most guardian models rely on rigid, predefined safety categories that fail to generalize across diverse linguistic and sociocultural contexts. Robust safety, therefore, requires flexible, runtime-enforceable policies and benchmarks that reflect local norms, harm scenarios, and cultural expectations. We introduce UbuntuGuard, the first African policy-based safety benchmark built from adversarial queries authored by 155 domain experts across sensitive fields, including healthcare. From these expert-crafted queries, we derive context-specific safety policies and reference responses that capture culturally grounded risk signals, enabling policy-aligned evaluation of guardian models. We evaluate 13 models, comprising six general-purpose LLMs and seven guardian models across three distinct variants: static, dynamic, and multilingual. Our findings reveal that existing English-centric benchmarks overestimate real-world multilingual safety, cross-lingual transfer provides partial but insufficient coverage, and dynamic models, while better equipped to leverage policies at inference time, still struggle to fully localize African-language contexts. These findings highlight the urgent need for multilingual, culturally grounded safety benchmarks to enable the development of reliable and equitable guardian models for low-resource languages. Our code can be found online.\footnote{Code repository available at https://github.com/hemhemoh/UbuntuGuard.

Related papers

• SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs [56.34354368047577]
  Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts.<n>We introduce SurakshaEval, a novel safety benchmark composed of human-written prompts spanning real-world scenarios.<n>We benchmark a broad range of state-of-the-art LLMs on SurakshaEval, establish baseline safety performance, and identify recurring failure modes.
  arXiv  Detail & Related papers  (2026-08-08T02:03:33Z)
• LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review [9.895109697164283]
  Large Language Models (LLMs) have achieved substantial progress in safety alignment.<n>Their safety guarantees remain significantly weaker in low-resource and multilingual settings.
  arXiv  Detail & Related papers  (2026-07-20T21:11:00Z)
• IndicGuard: A Multilingual Safety Guard Model and Dataset for Indic Languages [2.584263027095689]
  We introduce IndicGuard, a multilingual safety guard model and dataset for Indic languages.<n>We construct a high-volume, culturally nuanced safety dataset encompassing ten major Indic languages.<n>We fine-tune a 4B- parameter instruction-tuned model based on Gemma-3-4B-IT to serve as a multilingual safety guardrail.
  arXiv  Detail & Related papers  (2026-06-22T04:36:57Z)
• Schützen: Evaluating LLM Safety in Bulgarian and German Contexts [53.865251738592605]
  This paper introduces Schtzen: a German--Bulgarian safety dataset designed to assess model answerability under risk.<n> Experiments with multilingual and language-specific LLMs reveal pronounced cross-language differences in safety behavior.
  arXiv  Detail & Related papers  (2026-06-09T18:01:19Z)
• Multilingual Refusal Alignment for Safer Large Language Models [53.64286756804503]
  We investigate whether single-language alignment transfers cross-lingually, how language consistency is preserved during training, and the resulting trade-offs with general knowledge capabilities.<n>We introduce RefusEU, a novel refusal alignment dataset covering 12 European languages, including a dedicated test set for evaluating current state-of-the-art models.<n>Our controlled Direct Preference Optimization (DPO) experiments provide two key insights: aligning models exclusively in English is insufficient to ensure cross-lingual safety, even for the same harm categories, whereas training on multilingual datasets can improve safety without degrading general performance, as measured by the Global MMLU benchmark.
  arXiv  Detail & Related papers  (2026-04-24T09:29:14Z)
• LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety [88.98698230989186]
  Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities in low-resource languages.<n>We attribute this gap to a mismatch between language-agnostic semantic understanding ability and language-dominant safety alignment biased toward high-resource languages.<n>We propose Language-Agnostic Semantic Alignment (LASA), which anchors safety alignment directly in semantic bottlenecks.
  arXiv  Detail & Related papers  (2026-04-13T15:59:50Z)
• IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia [0.6978180153516672]
  We present the first systematic evaluation of large language models (LLMs) safety across 12 Indic languages.<n>Cross-language agreement is just 12.8%, and textttSAFE rate variance exceeds 17% across languages.<n>We release textscIndicSafe, the first benchmark to enable culturally informed safety evaluation for Indic deployments.
  arXiv  Detail & Related papers  (2026-03-18T16:54:07Z)
• Bridging the Multilingual Safety Divide: Efficient, Culturally-Aware Alignment for Global South Languages [8.667909336164465]
  Large language models (LLMs) are being deployed across the Global South.<n> Everyday use involves low-resource languages, code-mixing, and culturally specific norms.<n>Our aim is to make multilingual safety a core requirement-not an add-on-for equitable AI in underrepresented regions.
  arXiv  Detail & Related papers  (2026-02-14T19:56:40Z)
• Improving Methodologies for LLM Evaluations Across Global Languages [19.63570354411416]
  The exercise shows how safety behaviours can vary across languages.<n>It also generated insights for improving multilingual safety evaluations.<n>This work represents an initial step toward a shared framework for multilingual safety testing of advanced AI systems.
  arXiv  Detail & Related papers  (2026-01-22T07:18:08Z)
• SEA-SafeguardBench: Evaluating AI Safety in SEA Languages and Cultures [36.95168918567729]
  Existing multilingual safety benchmarks often rely on machine-translated English data.<n>We introduce SEA-SafeguardBench, the first human-verified safety benchmark for SEA.<n>It covers eight languages, 21,640 samples, across three subsets: general, in-the-wild, and content generation.
  arXiv  Detail & Related papers  (2025-12-05T07:57:57Z)
• Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages [57.059267233093465]
  Large Language Models (LLMs) have transformed natural language processing, but their safety mechanisms remain under-explored in low-resource, multilingual settings.<n>We introduce textsfSGToxicGuard, a novel dataset and evaluation framework for benchmarking LLM safety in Singapore's diverse linguistic context.<n>We conduct extensive experiments with state-of-the-art multilingual LLMs, and the results uncover critical gaps in their safety guardrails.
  arXiv  Detail & Related papers  (2025-09-18T08:14:34Z)
• LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models [22.273388934888278]
  Our dataset comprises 45k entries in 12 languages, ranging from Hungarian to Malay.<n>Our benchmark provides a comprehensive suite of metrics for in-depth safety evaluation.
  arXiv  Detail & Related papers  (2025-08-18T08:59:01Z)
• MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety [56.77103365251923]
  Large Language Models (LLMs) are susceptible to adversarial attacks such as jailbreaking.<n>This vulnerability is exacerbated in multilingual settings, where multilingual safety-aligned data is often limited.<n>We introduce a multilingual guardrail with reasoning for prompt classification.
  arXiv  Detail & Related papers  (2025-04-21T17:15:06Z)
• LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistencies [63.10843814055688]
  M-ALERT is a benchmark that evaluates the safety of Large Language Models in five languages.<n>M-ALERT includes 15k high-quality prompts per language, totaling 75k, with category-wise annotations.<n>Our experiments on 39 state-of-the-art LLMs highlight the importance of language-specific safety analysis.
  arXiv  Detail & Related papers  (2024-12-19T16:46:54Z)
• Arabic Dataset for LLM Safeguard Evaluation [62.96160492994489]
  This study explores the safety of large language models (LLMs) in Arabic with its linguistic and cultural complexities.<n>We present an Arab-region-specific safety evaluation dataset consisting of 5,799 questions, including direct attacks, indirect attacks, and harmless requests with sensitive words.
  arXiv  Detail & Related papers  (2024-10-22T14:12:43Z)
• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
  ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
  arXiv  Detail & Related papers  (2024-04-06T15:01:47Z)
• All Languages Matter: On the Multilingual Safety of Large Language Models [96.47607891042523]
  We build the first multilingual safety benchmark for large language models (LLMs) XSafety covers 14 kinds of commonly used safety issues across 10 languages that span several language families. We propose several simple and effective prompting methods to improve the multilingual safety of ChatGPT.
  arXiv  Detail & Related papers  (2023-10-02T05:23:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.