Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

• URL: http://arxiv.org/abs/2608.07786v1
• Date: Fri, 07 Aug 2026 22:18:02 GMT
• Title: Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space
• Abstract summary: Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines.<n>We ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage.<n>We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models.
• Score: 9.829283524934848
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and evolution. Understanding these relationships is important for model provenance, governance, and supply-chain integrity. In this work, we investigate the notion of LLM "biometrics" (analogous to human biometrics) to ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage. We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models. To characterize these relationships, we propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives: (i) spectral energy, captured by singular value distributions to encode global magnitude patterns, and (ii) subspace alignment, quantified via subspace deviations to capture directional geometry. Our analysis uncovers a clear hierarchy of structural similarity in weight space: spectral energy reliably distinguishes independently trained models and different model families, while subspace alignment enables fine-grained discrimination among closely related models, including variations in dataset scale and post-training procedures. Extensive experiments on over 110 diverse open-weight LLM pairs demonstrate that weight-space geometry provides a robust and interpretable signal for model lineage, enabling coarse-grained regime separation and fine-grained discrimination within shared-base models.

Related papers

• Spectral Signatures of Large Language Models [13.878105276531976]
  We propose spectral shape-based metrics for managing and quantifying large language models (LLMs)<n>Our approach uses the shape information of the weight empirical spectral density as a compact spectral signature of each model.<n>This signature captures intrinsic properties of pretrained models and remains robust during post-training, making it suitable for model-level analysis.
  arXiv  Detail & Related papers  (2026-07-03T14:30:22Z)
• SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representations [12.185380843937196]
  We introduce SEMASIA, a large-scale collection of latent representations extracted from approximately 1,700 pretrained vision models.<n>We analyze the conceptual organization of individual latent spaces, showing consistent prototype-like clustering.<n>We perform a large-scale regression analysis of how pretraining-data complexity, specialization, transfer learning, augmentation, and model scale relate to geometric and probing properties of embeddings.
  arXiv  Detail & Related papers  (2026-05-10T11:42:36Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Large Language Model Sourcing: A Survey [84.63438376832471]
  Large language models (LLMs) have revolutionized artificial intelligence, shifting from supporting objective tasks to empowering subjective decision-making.<n>Due to the black-box nature of LLMs and the human-like quality of their generated content, issues such as hallucinations, bias, unfairness, and copyright infringement become significant.<n>This survey presents a systematic investigation into provenance tracking for content generated by LLMs, organized around four interrelated dimensions.
  arXiv  Detail & Related papers  (2025-10-11T10:52:30Z)
• The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology [4.280045926995889]
  This study focuses on how adversarial inputs systematically affect the internal representation spaces of Large Language Models.<n>By quantifying the shape of activations and neuronal information flow, our architecture-agnostic framework reveals fundamental invariants of representational change.
  arXiv  Detail & Related papers  (2025-05-26T18:31:49Z)
• Multi-Scale Manifold Alignment for Interpreting Large Language Models: A Unified Information-Geometric Framework [4.935224714809964]
  We present Multi-Scale Manifold Alignment (MSMA), an information-geometric framework that decomposes LLM representations into local, intermediate, and global manifold.<n>We observe consistent hierarchical patterns and find that MSMA improves alignment metrics under multiple estimators.<n> Controlled interventions at different scales yield distinct and architecture-dependent effects on lexical diversity, sentence structure, and discourse coherence.
  arXiv  Detail & Related papers  (2025-05-24T10:25:58Z)
• Unraveling the Localized Latents: Learning Stratified Manifold Structures in LLM Embedding Space with Sparse Mixture-of-Experts [3.9426000822656224]
  We conjecture that in large language models, the embeddings live in a local manifold structure with different dimensions depending on the perplexities and domains of the input data.<n>By incorporating an attention-based soft-gating network, we verify that our model learns specialized sub-manifolds for an ensemble of input data sources.
  arXiv  Detail & Related papers  (2025-02-19T09:33:16Z)
• Aligning Large Language Models and Geometric Deep Models for Protein Representation [57.59506688299817]
  Latent representation alignment is used to map embeddings from different modalities into a shared space, often aligned with the embedding space of large language models (LLMs)<n>Preliminary protein-focused large language models (MLLMs) have emerged, but they have predominantly relied on approaches lacking a fundamental understanding of optimal alignment practices across representations.<n>In this study, we explore the alignment of multimodal representations between LLMs and Geometric Deep Models (GDMs) in the protein domain.<n>Our work examines alignment factors from both model and protein perspectives, identifying challenges in current alignment methodologies and proposing strategies to improve the alignment process.
  arXiv  Detail & Related papers  (2024-11-08T04:15:08Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• The Geometry of Self-supervised Learning Models and its Impact on Transfer Learning [62.601681746034956]
  Self-supervised learning (SSL) has emerged as a desirable paradigm in computer vision. We propose a data-driven geometric strategy to analyze different SSL models using local neighborhoods in the feature space induced by each.
  arXiv  Detail & Related papers  (2022-09-18T18:15:38Z)
• On the Symmetries of Deep Learning Models and their Internal Representations [1.418465438044804]
  We seek to connect the symmetries arising from the architecture of a family of models with the symmetries of that family's internal representation of data. Our work suggests that the symmetries of a network are propagated into the symmetries in that network's representation of data.
  arXiv  Detail & Related papers  (2022-05-27T22:29:08Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.