SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representations

• URL: http://arxiv.org/abs/2605.09485v1
• Date: Sun, 10 May 2026 11:42:36 GMT
• Title: SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representations
• Abstract summary: We introduce SEMASIA, a large-scale collection of latent representations extracted from approximately 1,700 pretrained vision models.<n>We analyze the conceptual organization of individual latent spaces, showing consistent prototype-like clustering.<n>We perform a large-scale regression analysis of how pretraining-data complexity, specialization, transfer learning, augmentation, and model scale relate to geometric and probing properties of embeddings.
• Score: 12.185380843937196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent representations learned by neural networks often exhibit semantic structure, where concept similarity is reflected by geometric proximity in embedding space. However, comparing such spaces across models remains difficult: changes in architecture, pretraining data, objective, or random seed can yield embeddings with similar content but incompatible geometry. This latent space alignment problem is central to interpretability, transfer and multimodal learning, federated systems, and semantic communication; however, progress remains limited by the lack of large-scale, model-diverse, and metadata-rich benchmarks. To address this gap, we introduce SEMASIA, a large-scale collection of latent representations extracted from approximately 1,700 pretrained vision models across eight standard image-classification benchmarks. SEMASIA pairs embeddings with structured metadata describing architectures, training regimes, pretraining sources, and model scale. We demonstrate three applications of the resource. First, we analyze the conceptual organization of individual latent spaces, showing consistent prototype-like clustering and hierarchical semantic neighborhoods across models and datasets. Second, we benchmark supervised alignment mappings between latent spaces using reconstruction error and downstream task performance. Third, we perform a large-scale regression analysis of how pretraining-data complexity, specialization, transfer learning, augmentation, and model scale relate to geometric and probing properties of embeddings. By coupling representational scale with standardized metadata, SEMASIA provides a reproducible foundation for studying latent geometry, evaluating alignment methods, and developing next-generation heterogeneous and interoperable AI systems.

Related papers

• Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer [55.982669669569624]
  This survey presents the first systematic review of foundation-model priors for hand-object interaction.<n>We establish a taxonomy of eight foundation-model sub-priors grouped into geometric, semantic, and visual families.<n>We examine how HOI-derived knowledge is used in robot learning, including human-data pretraining, human-to-robot skill transfer, and HOI-to-robot data generation.
  arXiv  Detail & Related papers  (2026-07-30T15:48:16Z)
• G$^2$TAM: Geometry Grounded Track Anything Model [49.05553245076823]
  We propose a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos.<n>G$2$TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views.<n>At its core is a cross-modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end-to-end spatial reconstruction and instance-consistent mask prediction.
  arXiv  Detail & Related papers  (2026-07-04T09:32:13Z)
• Uncovering the Latent Potential of Deep Intermediate Representations [16.988545433223933]
  We show that task-relevant information is distributed non-monotonically across layers and cannot be recovered by nave aggregation.<n>We introduce Layer-wise Optimal Embedding Selection (LOES), a constructive spectral method that identifies task-discriminative subspaces.<n>Our method reveals how semantic factors are distributed across layers, thereby enabling cross-lingual and cross-modal interpretability analyses.
  arXiv  Detail & Related papers  (2026-05-21T20:58:42Z)
• Pair2Scene: Learning Local Object Relations for Procedural Scene Generation [10.247549170637418]
  Pair2Scene is a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms.<n>We show that our framework outperforms existing methods in generating complex environments that go beyond training data.
  arXiv  Detail & Related papers  (2026-04-13T17:59:55Z)
• Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs [81.43302841109349]
  Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs)<n>We propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development.
  arXiv  Detail & Related papers  (2026-04-12T06:24:07Z)
• Hierarchical Neural Semantic Representation for 3D Semantic Correspondence [72.8101601086805]
  We design the hierarchical neural semantic representation (HNSR), which consists of a global semantic feature to capture high-level structure and multi-resolution local geometric features.<n>Second, we design a progressive global-to-local matching strategy, which establishes coarse semantic correspondence using the global semantic feature.<n>Third, our framework is training-free and broadly compatible with various pre-trained 3D generative backbones, demonstrating strong generalization across diverse shape categories.
  arXiv  Detail & Related papers  (2025-09-22T07:23:07Z)
• Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition [63.55828203989405]
  We introduce a novel Topology-Aware Modeling (TAM) framework for Sim2Real UDA on object point clouds.<n>Our approach mitigates the domain gap by leveraging global spatial topology, characterized by low-level, high-frequency 3D structures.<n>We propose an advanced self-training strategy that combines cross-domain contrastive learning with self-training.
  arXiv  Detail & Related papers  (2025-06-26T11:53:59Z)
• Connecting Neural Models Latent Geometries with Relative Geodesic Representations [21.71782603770616]
  We show that when a latent structure is shared between distinct latent spaces, relative distances between representations can be preserved, up to distortions.<n>We assume that distinct neural models parametrize approximately the same underlying manifold, and introduce a representation based on the pullback metric.<n>We validate our method on model stitching and retrieval tasks, covering autoencoders and vision foundation discriminative models.
  arXiv  Detail & Related papers  (2025-06-02T12:34:55Z)
• Learning Abstract World Models with a Group-Structured Latent Space [12.685414866379366]
  We show how geometric priors can be imposed on the representation manifold of a learned transition model.<n>We show experimentally that this leads to better predictions of the latent transition model than fully unstructured approaches.
  arXiv  Detail & Related papers  (2025-06-02T10:43:18Z)
• Interpretable deformable image registration: A geometric deep learning perspective [9.13809412085203]
  We present a theoretical foundation for designing an interpretable registration framework.<n>We formulate an end-to-end process that refines transformations in a coarse-to-fine fashion.<n>We conclude by showing significant improvement in performance metrics over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-12-17T19:47:10Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• On Characterizing the Evolution of Embedding Space of Neural Networks using Algebraic Topology [9.537910170141467]
  We study how the topology of feature embedding space changes as it passes through the layers of a well-trained deep neural network (DNN) through Betti numbers. We demonstrate that as depth increases, a topologically complicated dataset is transformed into a simple one, resulting in Betti numbers attaining their lowest possible value.
  arXiv  Detail & Related papers  (2023-11-08T10:45:12Z)
• GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning [55.79997930181418]
  Generalized Zero-Shot Learning aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. We propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation.
  arXiv  Detail & Related papers  (2022-07-05T04:04:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.