Visual Fingerprints for LLM Generation Comparison

• URL: http://arxiv.org/abs/2605.06054v1
• Date: Thu, 07 May 2026 11:40:12 GMT
• Title: Visual Fingerprints for LLM Generation Comparison
• Abstract summary: Large language model (LLM) outputs arise from complex interactions among prompts, system instructions, model parameters, and architecture.<n>We present an approach to visually compare LLM outputs across generation conditions by modeling responses as collections of linguistic choices.<n>We then visualize these distributions as visual fingerprints, enabling direct, distribution-level comparison of condition-specific tendencies.
• Score: 5.471433082052042
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) outputs arise from complex interactions among prompts, system instructions, model parameters, and architecture. We refer to specific configurations of these factors as generation conditions, each of which can bias outputs in various ways. Understanding how different generation conditions shape model behaviors is essential for tasks such as prompt design and model evaluation, yet it remains challenging due to the stochastic and open-ended nature of text generation. We present an approach to visually compare LLM outputs across generation conditions by modeling responses as collections of linguistic choices, including content, expression, and structure. We extract these choices using natural language processing pipelines and represent their distributions across repeated samples. We then visualize these distributions as visual fingerprints, enabling direct, distribution-level comparison of condition-specific tendencies. Through four usage scenarios, we demonstrate how visual fingerprints reveal consistent patterns in LLM behavior that are difficult to observe through individual responses or aggregate metrics.

Related papers

• ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding [23.27523995613204]
  We propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations.<n>By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns.<n>Our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.
  arXiv  Detail & Related papers  (2026-04-20T04:55:11Z)
• Addressing Explainability of Generative AI using SMILE (Statistical Model-agnostic Interpretability with Local Explanations) [0.0]
  This thesis introduces gSMILE, a unified framework for the explainability of generative models.<n>gSMILE employs controlled perturbations of textual input, Wasserstein distance metrics, and weighted surrogate modelling.<n>It produces robust, human-aligned attributions and generalises effectively across state-of-the-art generative models.
  arXiv  Detail & Related papers  (2026-02-01T12:54:51Z)
• Natural Fingerprints of Large Language Models [19.87526607747389]
  We show that even when large language models are trained on exactly the same dataset, their outputs remain distinguishable.<n>We refer to these unintended, distinctive characteristics as natural fingerprints.<n>These results suggest that training dynamics can systematically shape model behavior, independent of data or architecture.
  arXiv  Detail & Related papers  (2025-04-21T05:48:52Z)
• Deterministic or probabilistic? The psychology of LLMs as random number generators [0.0]
  Large Language Models (LLMs) have transformed text generation through inherently probabilistic context-aware mechanisms.<n>Our results reveal that, despite their transformers-based architecture, these models often exhibit deterministic responses when prompted for random numerical outputs.
  arXiv  Detail & Related papers  (2025-02-27T10:45:27Z)
• Multi-modal Auto-regressive Modeling via Visual Words [96.25078866446053]
  We propose the concept of visual tokens, which maps the visual features to probability distributions over Large Multi-modal Models' vocabulary. We further explore the distribution of visual features in the semantic space within LMM and the possibility of using text embeddings to represent visual information.
  arXiv  Detail & Related papers  (2024-03-12T14:58:52Z)
• Meta-Task Prompting Elicits Embeddings from Large Language Models [54.757445048329735]
  We introduce a new unsupervised text embedding method, Meta-Task Prompting with Explicit One-Word Limitation. We generate high-quality sentence embeddings from Large Language Models without the need for model fine-tuning. Our findings suggest a new scaling law, offering a versatile and resource-efficient approach for embedding generation across diverse scenarios.
  arXiv  Detail & Related papers  (2024-02-28T16:35:52Z)
• Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Models [18.03259038587496]
  In-context learning can be significantly influenced by the order of in-context demonstration examples. We introduce an unsupervised fine-tuning method, termed the Information-Augmented and Consistency-Enhanced approach. Our proposed method can reduce the sensitivity of CausalLMs to the order of in-context examples and exhibit robust generalizability.
  arXiv  Detail & Related papers  (2024-02-23T22:39:12Z)
• Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
  Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
  arXiv  Detail & Related papers  (2024-02-05T03:25:04Z)
• Revisiting Demonstration Selection Strategies in In-Context Learning [66.11652803887284]
  Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL) In this work, we first revisit the factors contributing to this variance from both data and model aspects, and find that the choice of demonstration is both data- and model-dependent. We propose a data- and model-dependent demonstration selection method, textbfTopK + ConE, based on the assumption that textitthe performance of a demonstration positively correlates with its contribution to the model's understanding of the test samples.
  arXiv  Detail & Related papers  (2024-01-22T16:25:27Z)
• Towards More Unified In-context Visual Understanding [74.55332581979292]
  We present a new ICL framework for visual understanding with multi-modal output enabled. First, we quantize and embed both text and visual prompt into a unified representational space. Then a decoder-only sparse transformer architecture is employed to perform generative modeling on them.
  arXiv  Detail & Related papers  (2023-12-05T06:02:21Z)
• FLIP: Fine-grained Alignment between ID-based Models and Pretrained Language Models for CTR Prediction [49.510163437116645]
  Click-through rate (CTR) prediction plays as a core function module in personalized online services. Traditional ID-based models for CTR prediction take as inputs the one-hot encoded ID features of tabular modality. Pretrained Language Models(PLMs) has given rise to another paradigm, which takes as inputs the sentences of textual modality. We propose to conduct Fine-grained feature-level ALignment between ID-based Models and Pretrained Language Models(FLIP) for CTR prediction.
  arXiv  Detail & Related papers  (2023-10-30T11:25:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.