Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contexts

• URL: http://arxiv.org/abs/2604.18091v1
• Date: Mon, 20 Apr 2026 11:08:33 GMT
• Title: Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contexts
• Abstract summary: We introduce a new multimodal generation task, culture-aware humorous captioning, which requires a model to generate a humorous caption conditioned on both an input image and a target cultural context.<n>We propose a six-dimensional evaluation framework covering image relevance, contextual fit, semantic richness, reasonableness, humor, and creativity.<n> Experimental results show that our method achieves stronger overall performance under the proposed evaluation framework, with particularly large gains in contextual fit and a better balance between image relevance and humor under cultural constraints.
• Score: 10.798870030280012
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent multimodal large language models have shown promising ability in generating humorous captions for images, yet they still lack stable control over explicit cultural context, making it difficult to jointly maintain image relevance, contextual appropriateness, and humor quality under a specified cultural background. To address this limitation, we introduce a new multimodal generation task, culture-aware humorous captioning, which requires a model to generate a humorous caption conditioned on both an input image and a target cultural context. Captions generated under different cultural contexts are not expected to share the same surface form, but should remain grounded in similar visual situations or humorous rationales.To support this task, we establish a six-dimensional evaluation framework covering image relevance, contextual fit, semantic richness, reasonableness, humor, and creativity. We further propose a staged alignment framework that first initializes the model with high-resource supervision under the Western cultural context, then performs multi-dimensional preference alignment via judge-based GRPO with a Degradation-aware Prototype Repulsion Constraint to mitigate reward hacking in open-ended generation, and finally adapts the model to the Eastern cultural context with a small amount of supervision. Experimental results show that our method achieves stronger overall performance under the proposed evaluation framework, with particularly large gains in contextual fit and a better balance between image relevance and humor under cultural constraints.

Related papers

• BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset [2.8823932597429196]
  Internet memes present a challenging setting where meaning emerges from implicit interactions between image, overlaid text, sarcasm, and socio-cultural knowledge.<n>We introduce BanglaMemeX, a culturally grounded multimodal benchmark comprising 3,000 Bangla memes annotated with multi-dimensional labels.<n>Our results highlight the need for culturally-aware multimodal systems capable of grounded reasoning under linguistic and cultural distribution shift.
  arXiv  Detail & Related papers  (2026-09-07T22:27:18Z)
• Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation [43.352493955825736]
  We show that current T2I models often produce culturally neutral or English-biased results under multilingual prompts.<n>We propose a probing method that localizes culture-sensitive signals to a small set of neurons in a few fixed layers.
  arXiv  Detail & Related papers  (2025-11-21T14:40:50Z)
• Vision Language Models are Confused Tourists [31.85723694463742]
  We introduce ConfusedTourist, a novel cultural adversarial robustness suite designed to assess Vision-Language Models (VLMs)<n>Our experiments reveal a critical vulnerability, where accuracy drops heavily under simple image-stacking perturbations and even worsens with its image-generation-based variant.<n>These findings highlight a critical challenge: visual cultural concept mixing can substantially impair even state-of-the-art VLMs.
  arXiv  Detail & Related papers  (2025-11-21T07:14:46Z)
• CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation [47.54347964192231]
  We introduce CaMMT, a benchmark of over 5,800 triples of images along with parallel captions in English and regional languages.<n>Through automatic and human evaluations, we find that visual context generally improves translation quality.
  arXiv  Detail & Related papers  (2025-05-30T10:42:44Z)
• From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test [50.51344198689069]
  We extend the human-centered word association test (WAT) to assess the alignment of large language models with cross-cultural cognition.<n>To address culture preference, we propose CultureSteer, an innovative approach by embedding cultural-specific semantic associations directly within the model's internal representation space.
  arXiv  Detail & Related papers  (2025-05-24T07:05:10Z)
• Deconstructing Bias: A Multifaceted Framework for Diagnosing Cultural and Compositional Inequities in Text-to-Image Generative Models [3.6335172274433414]
  This paper benchmarks the Component Inclusion Score (CIS), a metric designed to evaluate the fidelity of image generation across cultural contexts.<n>We quantify biases in terms of compositional fragility and contextual misalignment, revealing significant performance gaps between Western and non-Western cultural prompts.
  arXiv  Detail & Related papers  (2025-04-05T06:17:43Z)
• Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement [2.089922606370409]
  We propose a novel approach, Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement (Culture-TRIP)<n>Our approach retrieves cultural contexts and visual details related to the culture nouns in the prompt.<n>It iteratively refines and evaluates the prompt based on a set of cultural criteria and large language models.
  arXiv  Detail & Related papers  (2025-02-24T06:56:56Z)
• XMeCap: Meme Caption Generation with Sub-Image Adaptability [53.2509590113364]
  Humor, deeply rooted in societal meanings and cultural details, poses a unique challenge for machines.<n>We introduce the textscXMeCap framework, which adopts supervised fine-tuning and reinforcement learning.<n>textscXMeCap achieves an average evaluation score of 75.85 for single-image memes and 66.32 for multi-image memes, outperforming the best baseline by 6.75% and 8.56%, respectively.
  arXiv  Detail & Related papers  (2024-07-24T10:51:46Z)
• An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance [53.974497865647336]
  We take a first step towards translating images to make them culturally relevant. We build three pipelines comprising state-of-the-art generative models to do the task. We conduct a human evaluation of translated images to assess for cultural relevance and meaning preservation.
  arXiv  Detail & Related papers  (2024-04-01T17:08:50Z)
• Training-Free Consistent Text-to-Image Generation [80.4814768762066]
  Text-to-image models can portray the same subject across diverse prompts. Existing approaches fine-tune the model to teach it new words that describe specific user-provided subjects. We present ConsiStory, a training-free approach that enables consistent subject generation by sharing the internal activations of the pretrained model.
  arXiv  Detail & Related papers  (2024-02-05T18:42:34Z)
• On the Cultural Gap in Text-to-Image Generation [75.69755281031951]
  One challenge in text-to-image (T2I) generation is the inadvertent reflection of culture gaps present in the training data. There is no benchmark to systematically evaluate a T2I model's ability to generate cross-cultural images. We propose a Challenging Cross-Cultural (C3) benchmark with comprehensive evaluation criteria, which can assess how well-suited a model is to a target culture.
  arXiv  Detail & Related papers  (2023-07-06T13:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.