The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models

• URL: http://arxiv.org/abs/2511.11435v1
• Date: Fri, 14 Nov 2025 16:03:10 GMT
• Title: The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models
• Authors: Maria-Teresa De Rosa Palmini, Eva Cetinic,
• Abstract summary: We evaluate five diffusion models across 767 Wikidata-derived cultural references spanning static and dynamic imagery.<n>Our work reveals that the value of diffusion models lies not only in what they reproduce but in how they transform and recontextualize cultural knowledge.
• Score: 2.9793019246605676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Our work addresses the ambiguity between generalization and memorization in text-to-image diffusion models, focusing on a specific case we term multimodal iconicity. This refers to instances where images and texts evoke culturally shared associations, such as when a title recalls a familiar artwork or film scene. While prior research on memorization and unlearning emphasizes forgetting, we examine what is remembered and how, focusing on the balance between recognizing cultural references and reproducing them. We introduce an evaluation framework that separates recognition, whether a model identifies a reference, from realization, how it depicts it through replication or reinterpretation, quantified through measures capturing both dimensions. By evaluating five diffusion models across 767 Wikidata-derived cultural references spanning static and dynamic imagery, we show that our framework distinguishes replication from transformation more effectively than existing similarity-based methods. To assess linguistic sensitivity, we conduct prompt perturbation experiments using synonym substitutions and literal image descriptions, finding that models often reproduce iconic visual structures even when textual cues are altered. Finally, our analysis shows that cultural alignment correlates not only with training data frequency, but also textual uniqueness, reference popularity, and creation date. Our work reveals that the value of diffusion models lies not only in what they reproduce but in how they transform and recontextualize cultural knowledge, advancing evaluation beyond simple text-image matching toward richer contextual understanding.

Related papers

• Contrasting Cognitive Styles in Vision-Language Models: Holistic Attention in Japanese Versus Analytical Focus in English [4.8310710966636545]
  We investigate whether Vision-Language Models (VLMs) trained predominantly on different languages, specifically Japanese and English, exhibit similar culturally grounded attentional patterns.<n>Our findings suggest that VLMs not only internalize the structural properties of language but also reproduce cultural behaviors embedded in the training data, indicating that cultural cognition may implicitly shape model outputs.
  arXiv  Detail & Related papers  (2025-07-01T11:56:45Z)
• Quantifying Cross-Modality Memorization in Vision-Language Models [86.82366725590508]
  We study the unique characteristics of cross-modality memorization and conduct a systematic study centered on vision-language models.<n>Our results reveal that facts learned in one modality transfer to the other, but a significant gap exists between recalling information in the source and target modalities.
  arXiv  Detail & Related papers  (2025-06-05T16:10:47Z)
• An Information-Theoretic Approach to Identifying Formulaic Clusters in Textual Data [2.977406733413627]
  Formulaic texts, characterized by repetition and constrained expression, tend to have lower variability in self-information.<n>This study aims to identify formulaic clusters by analyzing recurring phrases, syntactic structures, and stylistic markers.<n>We develop an information-theoretic algorithm leveraging weighted self-information distributions to detect structured patterns in text.
  arXiv  Detail & Related papers  (2025-03-10T13:24:46Z)
• Diffusion Models Through a Global Lens: Are They Culturally Inclusive? [15.991121392458748]
  We introduce CultDiff benchmark, evaluating state-of-the-art diffusion models.<n>We show that these models often fail to generate cultural artifacts in architecture, clothing, and food, especially for underrepresented country regions.<n>We develop a neural-based image-image similarity metric, namely, CultDiff-S, to predict human judgment on real and generated images with cultural artifacts.
  arXiv  Detail & Related papers  (2025-02-13T03:05:42Z)
• Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models [65.82564074712836]
  We introduce DIFfusionHOI, a new HOI detector shedding light on text-to-image diffusion models. We first devise an inversion-based strategy to learn the expression of relation patterns between humans and objects in embedding space. These learned relation embeddings then serve as textual prompts, to steer diffusion models generate images that depict specific interactions.
  arXiv  Detail & Related papers  (2024-10-26T12:00:33Z)
• An Inversion-based Measure of Memorization for Diffusion Models [37.9715620828388]
  diffusion models are susceptible to training data memorization, raising concerns regarding copyright infringement and privacy invasion.<n>We introduce InvMM, an inversion-based measure of memorization, which is based on inverting a sensitive latent noise distribution accounting for the replication of an image.<n>InvMM is commensurable between samples, reveals the true extent of memorization from an adversarial standpoint and implies how memorization differs from membership.
  arXiv  Detail & Related papers  (2024-05-09T15:32:00Z)
• Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention [62.671435607043875]
  Research indicates that text-to-image diffusion models replicate images from their training data, raising tremendous concerns about potential copyright infringement and privacy risks.<n>We reveal that during memorization, the cross-attention tends to focus disproportionately on the embeddings of specific tokens.<n>We introduce an innovative approach to detect and mitigate memorization in diffusion models.
  arXiv  Detail & Related papers  (2024-03-17T01:27:00Z)
• Composition and Deformance: Measuring Imageability with a Text-to-Image Model [8.008504325316327]
  We propose methods that use generated images to measure the imageability of single English words and connected text. We find high correlation between the proposed computational measures of imageability and human judgments of individual words. We discuss possible effects of model training and implications for the study of compositionality in text-to-image models.
  arXiv  Detail & Related papers  (2023-06-05T18:22:23Z)
• Perceptual Grouping in Contrastive Vision-Language Models [59.1542019031645]
  We show how vision-language models are able to understand where objects reside within an image and group together visually related parts of the imagery. We propose a minimal set of modifications that results in models that uniquely learn both semantic and spatial information.
  arXiv  Detail & Related papers  (2022-10-18T17:01:35Z)
• Cross-Modal Coherence for Text-to-Image Retrieval [35.82045187976062]
  We train a Cross-Modal Coherence Modelfor text-to-image retrieval task. Our analysis shows that models trained with image-text coherence relations can retrieve images originally paired with target text more often than coherence-agnostic models. Our findings provide insights into the ways that different modalities communicate and the role of coherence relations in capturing commonsense inferences in text and imagery.
  arXiv  Detail & Related papers  (2021-09-22T21:31:27Z)
• Consensus-Aware Visual-Semantic Embedding for Image-Text Matching [69.34076386926984]
  Image-text matching plays a central role in bridging vision and language. Most existing approaches only rely on the image-text instance pair to learn their representations. We propose a Consensus-aware Visual-Semantic Embedding model to incorporate the consensus information.
  arXiv  Detail & Related papers  (2020-07-17T10:22:57Z)
• Probing Contextual Language Models for Common Ground with Visual Representations [76.05769268286038]
  We design a probing model that evaluates how effective are text-only representations in distinguishing between matching and non-matching visual representations. Our findings show that language representations alone provide a strong signal for retrieving image patches from the correct object categories. Visually grounded language models slightly outperform text-only language models in instance retrieval, but greatly under-perform humans.
  arXiv  Detail & Related papers  (2020-05-01T21:28:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.