Beyond ICA: Identifiability by Symmetry Breaking

• URL: http://arxiv.org/abs/2607.23182v1
• Date: Sat, 25 Jul 2026 12:27:10 GMT
• Title: Beyond ICA: Identifiability by Symmetry Breaking
• Abstract summary: We prove the identifiability of deep generative models (DGMs) with piecewise-affine (PWA) decoders and Gaussian mixture model (GMM) priors.<n>We introduce three algebraic contrast principles for symmetry breaking.<n>Results form a hierarchy: from law identifiability (LID; latent distribution up to a global affine map) through map identifiability (MID; decoder up to the same map) to posterior and pointwise identifiability.
• Score: 0.15229257192293197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We prove the identifiability of deep generative models (DGMs) with piecewise-affine (PWA) decoders and Gaussian mixture model (GMM) priors, in a purely unsupervised setting. We introduce three algebraic contrast principles for symmetry breaking: domain contrast, which trivializes the mixture symmetry group; mechanism contrast, which ensures every decoder branch is witnessed by a unique boundary; and interaction contrast, which forbids parameter conspiracies between latent components and decoder branches. Together they exploit the interplay between the discrete combinatorics of the PWA map and the continuous symmetry structure of the latent GMM. Continuity is replaced by algebraic symmetry conditions; injectivity is decoupled from structural identification and required only for pointwise inversion. Our results form a hierarchy: from law identifiability (LID; latent distribution up to a global affine map) through map identifiability (MID; decoder up to the same map) to posterior and pointwise identifiability. The ICA-form ambiguity emerges under conditions on diagonal component covariances. Assumptions are only on the data-generating process, not on learning methods, except for the interaction contrast. To our knowledge this is the first to make algebraic symmetry-breaking the engine of nonlinear identifiability, the first to admit discontinuous decoders, and the first to handle fully non-injective decoders, where every observation admits multiple latent codes.

Related papers

• Deep Skew-t Mixture Models [2.5794915063815664]
  High-dimensional clustering is challenging when component distributions are both heavy-tailed and directionally asymmetric.<n>We propose a hierarchical factor-analytic mixture based on the generalised-hyperbolic skew-$t$ normal mean--variance representation.<n>A shared inverse-gamma mixing variable is propagated along each complete latent pathway, allowing heavy tails and directional asymmetry to be modelled jointly.
  arXiv  Detail & Related papers  (2026-09-01T06:02:11Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• A Doeblin-Anchored Contrastive Chart for Learning Markov Transition Kernels [1.2691047660244335]
  This paper introduces a statistical-to-dynamical coordinate framework for learning transition kernels from contrastive objectives.<n>We prove that the anchored contrastive risk identifies the anchored transition density and calibrates excess risk to density error.
  arXiv  Detail & Related papers  (2026-06-01T13:26:44Z)
• High-Dimensional Latents Should Be Diagnosed Through Phase Structure [56.362776482614976]
  We study autoencoder and variational-autoencoder latent spaces through the lens of spin-glass theory.<n>We formalize a latent-space spin-glass dictionary for a fixed decoder.<n>We show that driving the latent system toward the edge-of-stability of the topological trivialization regime has concrete downstream consequences.
  arXiv  Detail & Related papers  (2026-05-23T06:17:23Z)
• The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning [0.5854803320592717]
  We prove optimality in a linear-Gaussian model (Thm. A), necessity of range coverage for any quadratic penalty that zeros deployment drift (Thm. G)<n>Thirteen blocks (ML through Qwen2.5-7B) test matched vs isotropic vs wrong-direction penalties on geometry and deployment drift.<n>We claim a falsifiable design recipe: estimate Sigma_task, match Sigma', run the controls, report task and geometry separately.
  arXiv  Detail & Related papers  (2026-05-21T17:53:28Z)
• NOETHER: A Constructive Framework for Metamorphic Pattern Discovery from Operator Algebras [6.691613382732758]
  Metamorphic Testing is recognised in IEEE/ISO software-testing standards and increasingly recommended for AI systems.<n>Existing approaches share an inductive grounding that leaves three foundational questions open.<n>We propose a framework whose downstream step from operator algebra to MetaPattern set is mechanical and provable.
  arXiv  Detail & Related papers  (2026-05-17T11:18:35Z)
• DC-Merge: Improving Model Merging with Directional Consistency [62.02490833158024]
  DC-Merge is a method for directional-consistent model merging.<n>It balances the energy distribution of each task vector by smoothing its singular values.<n>Experiments on vision and vision-language benchmarks show that DC-Merge consistently achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-06T13:04:25Z)
• Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective [16.942478643768144]
  Masked Diffusion Models (MDMs) significantly accelerate inference by trading off sequential determinism.<n>We provide a unified information-theoretic framework to decouple and analyze two fundamental sources of failure: order sensitivity and parallelization bias.
  arXiv  Detail & Related papers  (2026-01-30T20:15:18Z)
• DiffGRM: Diffusion-based Generative Recommendation Model [63.35379395455103]
  Generative recommendation (GR) is an emerging paradigm that represents each item via a tokenizer as an n-digit semantic ID (SID)<n>We propose DiffGRM, a diffusion-based GR model that replaces the autoregressive decoder with a masked discrete diffusion model (MDM)<n> Experiments show consistent gains over strong generative and discriminative recommendation baselines on multiple datasets.
  arXiv  Detail & Related papers  (2025-10-21T03:23:32Z)
• Ordinal Label-Distribution Learning with Constrained Asymmetric Priors for Imbalanced Retinal Grading [9.147336466586017]
  Diabetic retinopathy grading is inherently ordinal and long-tailed.<n>We propose the Constrained Asymmetric Prior Wasserstein Autoencoder (CAP-WAE)<n>CAP-WAE consistently achieves state-of-the-art Quadratic Weighted Kappa, accuracy, and macro-F1.
  arXiv  Detail & Related papers  (2025-09-30T11:58:49Z)
• Dense Semantic Matching with VGGT Prior [49.42199006453071]
  We propose an approach that retains VGGT's intrinsic strengths by reusing early feature stages, fine-tuning later ones, and adding a semantic head for bidirectional correspondences.<n>Our approach achieves superior geometry awareness, matching reliability, and manifold preservation, outperforming previous baselines.
  arXiv  Detail & Related papers  (2025-09-25T14:56:11Z)
• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.