BANGLASOCIALBENCH: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interaction

• URL: http://arxiv.org/abs/2603.15949v1
• Date: Mon, 16 Mar 2026 21:58:32 GMT
• Title: BANGLASOCIALBENCH: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interaction
• Abstract summary: We introduce BANGLASOCIALBENCH, the first benchmark designed to evaluate sociopragmatic competence in Bangla through context-dependent language use.<n>The benchmark spans three domains: Bangla Address Terms, Kinship Reasoning, and Social Customs, and consists of 1,719 culturally grounded instances written and verified by native Bangla speakers.<n>Our findings show that sociopragmatic failures are often structured and non-random, revealing persistent limitations in how current LLMs infer and apply culturally appropriate language use.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models have demonstrated strong multilingual fluency, yet fluency alone does not guarantee socially appropriate language use. In high-context languages, communicative competence requires sensitivity to social hierarchy, relational roles, and interactional norms that are encoded directly in everyday language. Bangla exemplifies this challenge through its three-tiered pronominal system, kinship-based addressing, and culturally embedded social customs. We introduce BANGLASOCIALBENCH, the first benchmark designed to evaluate sociopragmatic competence in Bangla through context-dependent language use rather than factual recall. The benchmark spans three domains: Bangla Address Terms, Kinship Reasoning, and Social Customs, and consists of 1,719 culturally grounded instances written and verified by native Bangla speakers. We evaluate twelve contemporary LLMs in a zero-shot setting and observe systematic patterns of cultural misalignment. Models frequently default to overly formal address forms, fail to recognize multiple socially acceptable address pronouns, and conflate kinship terminology across religious contexts. Our findings show that sociopragmatic failures are often structured and non-random, revealing persistent limitations in how current LLMs infer and apply culturally appropriate language use in realistic Bangladeshi social interactions.

Related papers

• The Shape of Power: A Multilingual Framework for Social Power Reasoning in Dialogues [19.678644625110064]
  We introduce a theoretically grounded framework for studying social power in multilingual dialogue through movie screenplays.<n>We constructed an initial corpus containing 15,836 annotated instances from 100 scenes in French and Egyptian Arabic movies.<n>Our analysis reveals strong agreement on observable demographic and contextual attributes, while socially interpretive aspects, such as power asymmetry and intention alignment, remain more contested.
  arXiv  Detail & Related papers  (2026-08-28T10:05:26Z)
• When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models [17.6283905669333]
  Large language models (LLMs) are widely used as cross-lingual knowledge interfaces.<n>Culturally grounded questions often reflect globally dominant narratives rather than local contexts.<n>We introduce textttCulturalNB, a dataset of 717 manually curated Bengali cultural instances with parallel Bangla--English question--answer pairs.
  arXiv  Detail & Related papers  (2026-05-28T18:58:32Z)
• Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation [58.01855677487771]
  We show that multilingual large language models (MLLMs) frequently exhibit inconsistent behaviour when the prompt's language changes.<n>We propose Cross-lingual Cultural Consistent Preference optimisation (C-3PO), a consensus-driven alignment framework.<n>C-3PO achieves up to a 0.10-point absolute increase in $_S$ over unaligned models, outperforming strong prompting and representation steering baselines.
  arXiv  Detail & Related papers  (2026-04-02T14:04:06Z)
• Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects [1.6884607868633077]
  Built from 1,152 manually curated images across nine domains, the benchmark supports visual question answering and captioning.<n>Our experiments show that evaluating only standard Bangla overestimates true model capability.<n>Across domains, the main bottleneck is missing cultural knowledge rather than visual grounding alone.
  arXiv  Detail & Related papers  (2026-03-22T10:55:08Z)
• Culturally-Grounded Governance for Multilingual Language Models: Rights, Data Boundaries, and Accountable AI Design [2.5424331328233207]
  This paper synthesizes existing evidence on multilingual model behavior, data asymmetries, and sociotechnical harm.<n>We identify three interrelated governance challenges: cultural and linguistic inequities in training data and evaluation practices, misalignment between global deployment and locally situated norms, values, and power structures, and limited accountability mechanisms for addressing harms experienced by marginalized language communities.<n>We argue that culturally grounded governance is essential for ensuring that multilingual language models do not reproduce existing global inequalities under the guise of scale and neutrality.
  arXiv  Detail & Related papers  (2026-01-31T03:47:30Z)
• BengaliMoralBench: A Benchmark for Auditing Moral Reasoning in Large Language Models within Bengali Language and Culture [5.215285027585101]
  Bengali is spoken by over 285 million people and ranked 6th globally.<n>Existing ethics benchmarks are largely English-centric and shaped by Western frameworks.<n>We introduce BengaliMoralBench, the first large-scale ethics benchmark for the Bengali language and socio-cultural contexts.
  arXiv  Detail & Related papers  (2025-11-05T04:55:35Z)
• MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation [91.22008265721952]
  MMA-ASIA centers on a human-curated, multilingual, and multimodally aligned benchmark covering 8 Asian countries and 10 languages.<n>This is the first dataset aligned at the input level across three modalities: text, image (visual question answering), and speech.<n>We propose a five-dimensional evaluation protocol that measures: (i) cultural-awareness disparities across countries, (ii) cross-lingual consistency, (iii) cross-modal consistency, (iv) cultural knowledge generalization, and (v) grounding validity.
  arXiv  Detail & Related papers  (2025-10-07T14:12:12Z)
• NormGenesis: Multicultural Dialogue Generation via Exemplar-Guided Social Norm Modeling and Violation Recovery [4.701242219264098]
  We present NormGenesis, a framework for generating socially grounded dialogues in English, Chinese, and Korean.<n>We propose a novel dialogue type, Violation-to-Resolution (V2R), which models the progression of conversations following norm violations through recognition and socially appropriate repair.<n>We show that models trained on our V2R-augmented data exhibit improved pragmatic competence in ethically sensitive contexts.
  arXiv  Detail & Related papers  (2025-09-22T20:29:25Z)
• From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test [50.51344198689069]
  We extend the human-centered word association test (WAT) to assess the alignment of large language models with cross-cultural cognition.<n>To address culture preference, we propose CultureSteer, an innovative approach by embedding cultural-specific semantic associations directly within the model's internal representation space.
  arXiv  Detail & Related papers  (2025-05-24T07:05:10Z)
• Scalable Frame-based Construction of Sociocultural NormBases for Socially-Aware Dialogues [66.69453609603875]
  Sociocultural norms serve as guiding principles for personal conduct in social interactions. We propose a scalable approach for constructing a Sociocultural Norm (SCN) Base using Large Language Models (LLMs) We construct a comprehensive and publicly accessible Chinese Sociocultural NormBase.
  arXiv  Detail & Related papers  (2024-10-04T00:08:46Z)
• NormDial: A Comparable Bilingual Synthetic Dialog Dataset for Modeling Social Norm Adherence and Violation [18.605252945314724]
  We present a high-quality dyadic dialogue dataset with turn-by-turn annotations of social norm adherences and violations for Chinese and American cultures. Our dataset is synthetically generated in both Chinese and English using a human-in-the-loop pipeline.
  arXiv  Detail & Related papers  (2023-10-23T04:38:34Z)
• Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment [31.929550141633218]
  We propose a novel approach to discover and compare social norms across Chinese and American cultures. We build a high-quality dataset of 3,069 social norms aligned with social situations across Chinese and American cultures. To test the ability of models to reason about social norms across cultures, we introduce the task of explainable social norm entailment.
  arXiv  Detail & Related papers  (2023-05-23T19:43:47Z)
• NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly [61.77957329364812]
  We introduce a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery. NormSAGE elicits knowledge about norms through directed questions representing the norm discovery task and conversation context. It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct.
  arXiv  Detail & Related papers  (2022-10-16T18:30:05Z)
• Experience Grounds Language [185.73483760454454]
  Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates. Despite the incredible effectiveness of language processing models to tackle tasks after being trained on text alone, successful linguistic communication relies on a shared experience of the world.
  arXiv  Detail & Related papers  (2020-04-21T16:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.