A Fiber Criterion for Representation Identifiability in Supervised Learning

• URL: http://arxiv.org/abs/2606.01092v1
• Date: Sun, 31 May 2026 08:25:07 GMT
• Title: A Fiber Criterion for Representation Identifiability in Supervised Learning
• Abstract summary: Supervised learning evaluates predictors through their input-output behavior.<n>This paper formalizes the resulting representation-level identifiability problem.<n>The results clarify that representation-level claims require assumptions, objectives, measurements, or inductive biases beyond supervised predictive behavior alone.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Supervised learning evaluates predictors through their input-output behavior. When a predictor is implemented as a composition $f=c\circ h$, supervised evidence constrains the composite map $f$ but need not determine the representation-head factorization $(h,c)$. This paper formalizes the resulting representation-level identifiability problem: for a class of admissible representation-head pairs, a representation property is identifiable from the induced predictor exactly when it is constant on the fibers of the projection $(h,c)\mapsto c\circ h$, equivalently when it descends to a well-defined property of the predictor. Predictor-preserving augmentation gives a canonical obstruction: auxiliary information can be appended to a representation while the head ignores it, leaving the predictor unchanged but altering properties such as minimality, compression, invariance, equivariance, nuisance information, or semantic accessibility. This construction separates representation identifiability from optimization and finite-sample estimation. Finite-sample diagnostics illustrate, rather than prove, the criterion: exact algebraic witnesses hold the predictor fixed while changing representation diagnostics, and matched-performance Waterbirds models show that different constraints can select different representations at similar supervised performance. The results clarify that representation-level claims require assumptions, objectives, measurements, or inductive biases beyond supervised predictive behavior alone.

Related papers

• Lifted Model Construction under Approximate Commutativity [5.629677133940586]
  We show how -commutativity can be exploited for lifted model construction, downstream probabilistic inference, and prove strict bounds on the induced approximation error.<n>These theoretical guarantees are confirmed empirically, demonstrating comparable query accuracy at lower runtime.
  arXiv  Detail & Related papers  (2026-08-25T15:32:47Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• DiffImaginE: Imagine to Verify Entity Types with Diffusion [17.922112345279643]
  We introduce DiffImaginE, which formulates MNER type verification as conditional latent diffusion inference.<n>Given span-localised visual evidence, a type-conditioned denoiser predicts noise injected into its standardised latent.<n>The resulting denoising error provides an ELBO-consistent surrogate for type-conditional negative log-likelihood.
  arXiv  Detail & Related papers  (2026-08-04T02:13:45Z)
• What Makes a Representation Good for Single-Cell Perturbation Prediction? [37.84894703250901]
  PerturbedVAE is a general framework designed to resolve this signal imbalance.<n>We provide an identifiability analysis that characterizes the conditions under which perturbation effects can be reliably recovered.<n>PerturbedVAE achieves state-of-the-art performance on a widely used benchmark multiple evaluation settings.
  arXiv  Detail & Related papers  (2026-05-19T04:30:11Z)
• FACTOR: Counterfactual Training-Free Test-Time Adaptation for Open-Vocabulary Object Detection [63.91351553178842]
  FACTOR is a training-free Test-time adaptation for Open-vocabulary object detection.<n>It quantifies attribute sensitivity, semantic relevance, and prediction variation to selectively suppress attribute-dependent predictions.<n>Experiments on PASCAL-C, COCO-C, and FoggyCityscapes show that FACTOR consistently outperforms prior TTA methods.
  arXiv  Detail & Related papers  (2026-05-05T02:31:18Z)
• Enhancing Pre-trained Representation Classifiability can Boost its Interpretability [112.296393156262]
  We quantify the representation interpretability by leveraging its correlation with the ratio of interpretable semantics within the representations.<n>We propose the Inherent Interpretability Score (IIS) that evaluates the information loss, measures the ratio of interpretable semantics, and quantifies the representation interpretability.
  arXiv  Detail & Related papers  (2025-10-28T06:21:06Z)
• Emergence of Quantised Representations Isolated to Anisotropic Functions [0.0]
  This paper presents a novel methodology for determining representational structure, which builds upon the existing Spotlight Resonance method.<n>It shows how discrete representations can emerge and organise in autoencoder models, through a controlled ablation study in which only the activation function is altered.<n>Using this technique, the validity of whether function-driven symmetries can act as implicit inductive biases on representations is determined.
  arXiv  Detail & Related papers  (2025-07-16T09:27:54Z)
• I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data? [76.15163242945813]
  Large language models (LLMs) have led many to conclude that they exhibit a form of intelligence.<n>We introduce a novel generative model that generates tokens on the basis of human-interpretable concepts represented as latent discrete variables.
  arXiv  Detail & Related papers  (2025-03-12T01:21:17Z)
• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• Predictive Inference with Feature Conformal Prediction [80.77443423828315]
  We propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces. From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions. Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods.
  arXiv  Detail & Related papers  (2022-10-01T02:57:37Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.