Adaptive Originality Filtering: Rejection Based Prompting and RiddleScore for Culturally Grounded Multilingual Riddle Generation

• URL: http://arxiv.org/abs/2508.18709v3
• Date: Thu, 09 Oct 2025 04:11:04 GMT
• Title: Adaptive Originality Filtering: Rejection Based Prompting and RiddleScore for Culturally Grounded Multilingual Riddle Generation
• Abstract summary: We introduce Adaptive Originality Filtering to enforce novelty and cultural fidelity via semantic rejection.<n>RiddleScore is a metric combining novelty, diversity, fluency, and answer alignment.<n>Human evaluations confirm fluency, creativity, and cultural fit gains.
• Score: 8.152950449986623
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models are increasingly tested on multilingual creativity, demanding culturally grounded, abstract generations. Standard prompting methods often produce repetitive or shallow outputs. We introduce Adaptive Originality Filtering (AOF), a prompting strategy that enforces novelty and cultural fidelity via semantic rejection. To assess quality, we propose RiddleScore, a metric combining novelty, diversity, fluency, and answer alignment. AOF improves Distinct-2 (0.915 in Japanese), reduces Self-BLEU (0.177), and raises RiddleScore (up to +57.1% in Arabic). Human evaluations confirm fluency, creativity, and cultural fit gains. However, improvements vary: Arabic shows greater RiddleScore gains than Distinct-2; Japanese sees similar changes. Though focused on riddles, our method may apply to broader creative tasks. Overall, semantic filtering with composite evaluation offers a lightweight path to culturally rich generation without fine-tuning.

Related papers

• CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia [72.20656906184539]
  We introduce CultureConverse, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue.<n>The resulting CultureConverse-DS dataset contains 14,610 benchmark (evaluation) episodes and 274,295 oracle-guided (gold-mode) dialogues.<n>Human annotation experiments suggest that our evaluation framework is a sufficient proxy for human judgment.
  arXiv  Detail & Related papers  (2026-08-28T14:56:46Z)
• Beyond Surface Cues: Disentangling Sociocultural Signals in Multilingual LLMs [28.912535376913436]
  The study analyzes 89,253 outputs from 12 Multilingual LLMs in English, French, and Chinese.<n>Removing direct identity cues reduces identity-label prediction in English and Chinese, but has a much smaller effect in French.<n>The ability to identify the source language drops substantially after translation and again after masking names.
  arXiv  Detail & Related papers  (2026-08-24T09:32:16Z)
• Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning [11.498685324357384]
  Figurative language is deeply culturally embedded; fluent use requires cultural immersion.<n>We ask whether fine-tuning on cultural data improve figurative language understanding, and vice versa.<n>Our findings suggest that the relationship between culture and figurative language, though conceptually natural, is not straightforwardly captured through fine-tuning alone.
  arXiv  Detail & Related papers  (2026-08-18T22:25:17Z)
• CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity [67.74087963315213]
  CreativeInstruct is an instruction-tuning method that teaches large language models to balance creative, base-model-like generations with the quality of post-trained models.<n>We show that CreativeInstruct matches or exceeds the diversity of both multi-model baselines and distilled variants of their outputs.<n>We also show the benefits of creative models as a substrate for RL: GRPO applied to a CreativeInstruct checkpoint improves by 4% on AMC and 5% points on MATH.
  arXiv  Detail & Related papers  (2026-08-07T17:55:48Z)
• Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responses [28.3173238194554]
  We introduce CEDAR, a benchmark constructed entirely from scenarios capturing culturally underlinetextscElicited underlinetextscDistinct underlinetextscAffective underlinetextscResponses.<n>The resulting benchmark comprises 10,962 instances across seven languages and 14 fine-grained emotion categories, with each language including 400 multimodal and 1,166 text-only samples.
  arXiv  Detail & Related papers  (2026-01-19T13:04:26Z)
• Extending Beacon to Hindi: Cultural Adaptation Drives Cross-Lingual Sycophancy [0.0]
  Sycophancy is the tendency of language models to prioritize agreement with user preferences over principled reasoning.<n>We extend the Beacon single-turn forced-choice sycophancy diagnostic to Hindi through a controlled three-condition design.<n>Across all models, sycophancy rates are consistently higher for culturally adapted Hindi prompts than for English.
  arXiv  Detail & Related papers  (2026-01-19T12:03:52Z)
• CURVE: A Benchmark for Cultural and Multilingual Long Video Reasoning [58.73855961335903]
  CURVE (Cultural Understanding and Reasoning in Video Evaluation) is a challenging benchmark for multicultural and multilingual video reasoning.<n>It comprises high-quality, entirely human-generated annotations from diverse, region-specific cultural videos across 18 global locales.<n>Our evaluations reveal that SoTA Video-LLMs struggle significantly, performing substantially below human-level accuracy.
  arXiv  Detail & Related papers  (2026-01-15T18:15:06Z)
• COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processes [83.84578306665976]
  Large language models exhibit systematic deficiencies in creative writing, particularly in non-English contexts.<n>We present COIG-Writer, a novel Chinese creative writing dataset that captures both diverse outputs and their underlying thought processes.
  arXiv  Detail & Related papers  (2025-10-16T15:01:19Z)
• CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models [0.42970700836450487]
  We introduce CRaFT, an explanation-based multilingual evaluation framework designed to assess how large language models (LLMs) reason across cultural contexts.<n>We apply the framework to 50 culturally grounded questions from the World Values Survey, translated into Arabic, Bengali, and Spanish, and evaluate three models (GPT, DeepSeek, and FANAR) across over 2,100 answer-explanation pairs.<n>Results reveal significant cross-lingual variation in reasoning: Arabic reduces fluency, Bengali enhances it, and Spanish remains largely stable.
  arXiv  Detail & Related papers  (2025-10-15T18:49:10Z)
• Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models [55.14276067678253]
  This paper introduces a novel methodology for efficiently identifying inherent cross-lingual weaknesses in Large Language Models (LLMs)<n>We construct a new dataset of over 6,000 bilingual pairs across 16 languages using this methodology, demonstrating its effectiveness in revealing weaknesses even in state-of-the-art models.<n>Further experiments investigate the relationship between linguistic similarity and cross-lingual weaknesses, revealing that linguistically related languages share similar performance patterns.
  arXiv  Detail & Related papers  (2025-05-24T12:31:27Z)
• JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation [63.83457341009046]
  JMMMU (Japanese MMMU) is the first large-scale Japanese benchmark designed to evaluate LMMs on expert-level tasks based on the Japanese cultural context.<n>Using the CA subset, we observe performance drop in many LMMs when evaluated in Japanese, which is purely attributable to language variation.<n>By combining both subsets, we identify that some LMMs perform well on the CA subset but not on the CS subset, exposing a shallow understanding of the Japanese language that lacks depth in cultural understanding.
  arXiv  Detail & Related papers  (2024-10-22T17:59:56Z)
• Understanding and Mitigating Language Confusion in LLMs [76.96033035093204]
  We evaluate 15 typologically diverse languages with existing and newly-created English and multilingual prompts.<n>We find that Llama Instruct and Mistral models exhibit high degrees of language confusion.<n>We find that language confusion can be partially mitigated via few-shot prompting, multilingual SFT and preference tuning.
  arXiv  Detail & Related papers  (2024-06-28T17:03:51Z)
• Learnable Item Tokenization for Generative Recommendation [78.30417863309061]
  We propose LETTER (a LEarnable Tokenizer for generaTivE Recommendation), which integrates hierarchical semantics, collaborative signals, and code assignment diversity. LETTER incorporates Residual Quantized VAE for semantic regularization, a contrastive alignment loss for collaborative regularization, and a diversity loss to mitigate code assignment bias.
  arXiv  Detail & Related papers  (2024-05-12T15:49:38Z)
• Fine-tuning Language Models for Factuality [96.5203774943198]
  Large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations' In this work, we fine-tune language models to be more factual, without human labeling.
  arXiv  Detail & Related papers  (2023-11-14T18:59:15Z)
• Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text Generation [21.096737598952853]
  We propose a new decoding method called Fidelity-Enriched Contrastive Search (FECS) FECS promotes tokens that are semantically similar to the provided source while penalizing repetitiveness in the generated text. Results show that FECS consistently enhances faithfulness across various language model sizes while maintaining output diversity comparable to well-performing decoding algorithms.
  arXiv  Detail & Related papers  (2023-10-23T14:27:45Z)
• Reranking for Natural Language Generation from Logical Forms: A Study based on Large Language Models [47.08364281023261]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language generation. However, their output quality can be inconsistent, posing challenges for generating natural language from logical forms (LFs)
  arXiv  Detail & Related papers  (2023-09-21T17:54:58Z)
• Both the validity of the cultural tightness index and the association with creativity and order are spurious -- a comment on Jackson et al [77.34726150561087]
  Jackson et al. generate a linguistic index of cultural tightness based on the Google Books Ngram corpus. We show here that the methods used by Jackson et al. are neither suitable for testing the validity of the index nor for establishing possible relationships with creativity/order.
  arXiv  Detail & Related papers  (2022-01-26T08:32:44Z)
• Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs [28.16894664889912]
  We present a novel technique for zero-shot paraphrase generation. Key contribution is an end-to-end multilingual paraphrasing model that is trained using translated parallel corpora.
  arXiv  Detail & Related papers  (2021-10-25T19:33:38Z)
• Generating texts under constraint through discriminator-guided MCTS [1.3750624267664153]
  We formalize constrained generation as a tree exploration process guided by a discriminator. Using a discriminator to guide this generation, rather than fine-tuning the LM, allows to apply the constraint more finely and dynamically. We show that our methods achieves state-of-the-art results in constrained generation, without having to tune the language model.
  arXiv  Detail & Related papers  (2021-09-28T09:29:15Z)
• Lingxi: A Diversity-aware Chinese Modern Poetry Generation System [43.36560720793425]
  Lingxi is a diversity-aware Chinese modern poetry generation system. We propose nucleus sampling with randomized head (NS-RH) algorithm. We find that even when a large portion of filtered vocabulary is randomized, it can actually generate fluent poetry.
  arXiv  Detail & Related papers  (2021-08-27T03:33:28Z)
• Multilingual AMR-to-Text Generation [22.842874899794996]
  We create multilingual AMR-to-text models that generate in twenty one different languages. For eighteen languages, based on automatic metrics, our multilingual models surpass baselines that generate into a single language. We analyse the ability of our multilingual models to accurately capture morphology and word order using human evaluation, and find that native speakers judge our generations to be fluent.
  arXiv  Detail & Related papers  (2020-11-10T22:47:14Z)
• Automatic Dialect Adaptation in Finnish and its Effect on Perceived Creativity [2.231167375820083]
  We study the influence dialectal adaptation has on perceived creativity of computer generated poetry. The more the dialect deviates from the standard Finnish, the lower scores people tend to give on an existing evaluation metric.
  arXiv  Detail & Related papers  (2020-09-06T09:28:44Z)
• Inducing Language-Agnostic Multilingual Representations [61.97381112847459]
  Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world. We examine three approaches for this: (i) re-aligning the vector spaces of target languages to a pivot source language; (ii) removing language-specific means and variances, which yields better discriminativeness of embeddings as a by-product; and (iii) increasing input similarity across languages by removing morphological contractions and sentence reordering.
  arXiv  Detail & Related papers  (2020-08-20T17:58:56Z)
• MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space [79.70053419040902]
  We propose MixPoet, a novel model that absorbs multiple factors to create various styles and promote diversity. Based on a semi-supervised variational autoencoder, our model disentangles the latent space into some subspaces, with each conditioned on one influence factor by adversarial training. Experiment results on Chinese poetry demonstrate that MixPoet improves both diversity and quality against three state-of-the-art models.
  arXiv  Detail & Related papers  (2020-03-13T03:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.