PromptSplit: Revealing Prompt-Level Disagreement in Generative Models

• URL: http://arxiv.org/abs/2602.04009v2
• Date: Fri, 06 Feb 2026 11:34:05 GMT
• Title: PromptSplit: Revealing Prompt-Level Disagreement in Generative Models
• Authors: Mehdi Lotfian, Mohammad Jalali, Farzan Farnia,
• Abstract summary: Prompt-guided generative AI models have rapidly expanded across vision and language domains.<n>We propose PromptSplit, a kernel-based framework for detecting and analyzing prompt-dependent disagreement between generative models.<n>Experiments across text-to-image, text-to-text, and image-captioning settings demonstrate that PromptSplit accurately detects ground-truth behavioral differences.
• Score: 18.957478338649114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt-guided generative AI models have rapidly expanded across vision and language domains, producing realistic and diverse outputs from textual inputs. The growing variety of such models, trained with different data and architectures, calls for principled methods to identify which types of prompts lead to distinct model behaviors. In this work, we propose PromptSplit, a kernel-based framework for detecting and analyzing prompt-dependent disagreement between generative models. For each compared model pair, PromptSplit constructs a joint prompt--output representation by forming tensor-product embeddings of the prompt and image (or text) features, and then computes the corresponding kernel covariance matrix. We utilize the eigenspace of the weighted difference between these matrices to identify the main directions of behavioral difference across prompts. To ensure scalability, we employ a random-projection approximation that reduces computational complexity to $O(nr^2 + r^3)$ for projection dimension $r$. We further provide a theoretical analysis showing that this approximation yields an eigenstructure estimate whose expected deviation from the full-dimensional result is bounded by $O(1/r^2)$. Experiments across text-to-image, text-to-text, and image-captioning settings demonstrate that PromptSplit accurately detects ground-truth behavioral differences and isolates the prompts responsible, offering an interpretable tool for detecting where generative models disagree.

Related papers

• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Discovering Divergent Representations between Text-to-Image Models [87.40710629963264]
  We investigate when and how visual representations learned by two different generative models diverge.<n>We introduce CompCon, an evolutionary search algorithm that discovers visual attributes more prevalent in one model's output than the other.<n>We use CompCon to compare popular text-to-image models, finding divergent representations such as how PixArt depicts prompts mentioning loneliness with wet streets.
  arXiv  Detail & Related papers  (2025-09-10T19:07:55Z)
• Comateformer: Combined Attention Transformer for Semantic Sentence Matching [11.746010399185437]
  We propose a novel semantic sentence matching model named Combined Attention Network based on Transformer model (Comateformer)<n>In Comateformer model, we design a novel transformer-based quasi-attention mechanism with compositional properties.<n>Our proposed approach builds on the intuition of similarity and dissimilarity (negative affinity) when calculating dual affinity scores.
  arXiv  Detail & Related papers  (2024-12-10T06:18:07Z)
• Conditional Vendi Score: An Information-Theoretic Approach to Diversity Evaluation of Prompt-based Generative Models [15.40817940713399]
  We introduce the Conditional-Vendi score based on $H(X|T)$ to quantify the internal diversity of the model. We conduct several numerical experiments to show the correlation between the Conditional-Vendi score and the internal diversity of text-conditioned generative models.
  arXiv  Detail & Related papers  (2024-11-05T05:30:39Z)
• Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
  Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
  arXiv  Detail & Related papers  (2024-02-05T03:25:04Z)
• CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection [49.510604614688745]
  We propose a framework called CLIP-AD to leverage the zero-shot capabilities of the large vision-language model CLIP. We note opposite predictions and irrelevant highlights in the direct computation of the anomaly maps.
  arXiv  Detail & Related papers  (2023-11-01T11:39:22Z)
• Interpretable time series neural representation for classification purposes [3.1201323892302444]
  The proposed model produces consistent, discrete, interpretable, and visualizable representations. The experiments show that the proposed model yields, on average better results than other interpretable approaches on multiple datasets.
  arXiv  Detail & Related papers  (2023-10-25T15:06:57Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• Multivariate Representation Learning for Information Retrieval [31.31440742912932]
  We propose a new representation learning framework for dense retrieval. Instead of learning a vector for each query and document, our framework learns a multivariate distribution. We show that it can be seamlessly integrated into the existing approximate nearest neighbor algorithms.
  arXiv  Detail & Related papers  (2023-04-27T20:30:46Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• It's FLAN time! Summing feature-wise latent representations for interpretability [0.0]
  We propose a novel class of structurally-constrained neural networks, which we call FLANs (Feature-wise Latent Additive Networks) FLANs process each input feature separately, computing for each of them a representation in a common latent space. These feature-wise latent representations are then simply summed, and the aggregated representation is used for prediction.
  arXiv  Detail & Related papers  (2021-06-18T12:19:33Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.