IdEst: Assessing Self-Supervised Learning Representations via Intrinsic Dimension

• URL: http://arxiv.org/abs/2606.03338v1
• Date: Tue, 02 Jun 2026 08:47:17 GMT
• Title: IdEst: Assessing Self-Supervised Learning Representations via Intrinsic Dimension
• Abstract summary: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning meaningful representations from unlabeled data.<n>We propose IdEst, a method for estimating the ID of SSL representations via the Minimum Spanning Tree dimension estimator.<n>Our results highlight intrinsic dimensionality as a principled geometric proxy for assessing SSL representations, complementing standard supervised probing protocols.
• Score: 14.00877995955839
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning meaningful representations from unlabeled data. However, the standard protocol for evaluating these representations, linear probing, is computationally expensive, sensitive to hyperparameters, and provides limited insight into the geometric structure of the representation space. In this work, motivated by connections between neural network generalization and intrinsic dimension (ID) we propose IdEst, a method for estimating the ID of SSL representations via the Minimum Spanning Tree dimension estimator ($\mathrm{dim}_\mathrm{MST}$). Across diverse datasets, architectures, and SSL pretraining objectives, we show that IdEst strongly correlates with downstream linear probe performances. Furthermore, we demonstrate that IdEst enables efficient hyperparameter selection, significantly reducing the computational cost compared to supervised alternatives. Our results highlight intrinsic dimensionality as a principled geometric proxy for assessing SSL representations, complementing standard supervised probing protocols.

Related papers

• Self-Supervised Representation-Guided Generative Dataset Distillation [55.27114962330541]
  We propose self-supervised representation-guided generative dataset distillation (SRG)<n>SRG translates the SSL geometry into diffusion guidance.<n>It consistently outperforms evaluated generative baselines across multiple datasets and IPC settings.
  arXiv  Detail & Related papers  (2026-08-04T06:51:45Z)
• Generate in Reconstruction Space, Match in Semantic Space: Transport Geometry for One-Step Generation [16.138914735031744]
  Generative modeling and self-supervised representation learning (SSL) optimize structurally different objectives.<n>We study the benefits of SSL in generative modeling in the framework of one-step generation.<n>We find that this objective becomes highly effective when computed in a semantically structured SSL feature space.
  arXiv  Detail & Related papers  (2026-05-30T04:03:47Z)
• T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning [15.016777234800585]
  Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations without labeled data.<n>Recent studies have highlighted two pivotal properties for effective representations.<n>We introduce T-REGS, a simple regularization framework for SSL based on the length of the Minimum Spanning Tree (MST) over the learned representation.
  arXiv  Detail & Related papers  (2025-10-27T16:16:40Z)
• Exploring Structural Degradation in Dense Representations for Self-supervised Learning [84.52554180480037]
  We observe a counterintuitive phenomenon in self-supervised learning (SSL): longer training may impair the performance of dense prediction tasks.<n>We refer to this phenomenon as Self-supervised Dense Degradation (SDD) and demonstrate its consistent presence across sixteen state-of-the-art SSL methods.<n>We introduce a Dense representation Structure Estimator (DSE) composed of a class-relevance measure and an effective dimensionality measure.
  arXiv  Detail & Related papers  (2025-10-20T08:40:16Z)
• A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification [51.35500308126506]
  Self-supervised learning (SSL) is a machine learning approach where the data itself provides supervision, eliminating the need for external labels. We study how classification-based evaluation protocols for SSL correlate and how well they predict downstream performance on different dataset types.
  arXiv  Detail & Related papers  (2024-07-16T23:17:36Z)
• LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures [24.40012454562582]
  LiDAR is a metric designed to measure the quality of representations within Joint embedding architectures. Our proposed criterion presents a more robust and intuitive means of assessing the quality of representations within JE architectures.
  arXiv  Detail & Related papers  (2023-12-07T02:31:28Z)
• Proximal Symmetric Non-negative Latent Factor Analysis: A Novel Approach to Highly-Accurate Representation of Undirected Weighted Networks [2.1797442801107056]
  Undirected Weighted Network (UWN) is commonly found in big data-related applications. Existing models fail in either modeling its intrinsic symmetry or low-data density. Proximal Symmetric Nonnegative Latent-factor-analysis model is proposed.
  arXiv  Detail & Related papers  (2023-06-06T13:03:24Z)
• Deciphering the Projection Head: Representation Evaluation Self-supervised Learning [6.375931203397043]
  Self-supervised learning (SSL) aims to learn intrinsic features without labels. Projection head always plays an important role in improving the performance of the downstream task. We propose a Representation Evaluation Design (RED) in SSL models in which a shortcut connection between the representation and the projection vectors is built.
  arXiv  Detail & Related papers  (2023-01-28T13:13:53Z)
• Improving Self-Supervised Learning by Characterizing Idealized Representations [155.1457170539049]
  We prove necessary and sufficient conditions for any task invariant to given data augmentations. For contrastive learning, our framework prescribes simple but significant improvements to previous methods. For non-contrastive learning, we use our framework to derive a simple and novel objective.
  arXiv  Detail & Related papers  (2022-09-13T18:01:03Z)
• Deep Dimension Reduction for Supervised Representation Learning [51.10448064423656]
  We propose a deep dimension reduction approach to learning representations with essential characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero.
  arXiv  Detail & Related papers  (2020-06-10T14:47:43Z)
• Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
  Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method. PCL implicitly encodes semantic structures of the data into the learned embedding space. PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
  arXiv  Detail & Related papers  (2020-05-11T09:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.