From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flows

• URL: http://arxiv.org/abs/2602.06940v1
• Date: Fri, 06 Feb 2026 18:41:03 GMT
• Title: From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flows
• Authors: Daniel Galperin, Ullrich Köthe,
• Abstract summary: entropy-ordered flows (EOFlows) order latent dimensions by their explained entropy, analogous to PCA's explained variance.<n> EOFlows build on insights from Independent Mechanism Analysis, Principal Component Flows and Manifold Entropic Metrics.<n>We combine likelihood-based training with local Jacobian regularization and noise augmentation into a method that scales well to high-dimensional data such as images.
• Score: 8.351253396371686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning unsupervised representations that are both semantically meaningful and stable across runs remains a central challenge in modern representation learning. We introduce entropy-ordered flows (EOFlows), a normalizing-flow framework that orders latent dimensions by their explained entropy, analogously to PCA's explained variance. This ordering enables adaptive injective flows: after training, one may retain only the top C latent variables to form a compact core representation while the remaining variables capture fine-grained detail and noise, with C chosen flexibly at inference time rather than fixed during training. EOFlows build on insights from Independent Mechanism Analysis, Principal Component Flows and Manifold Entropic Metrics. We combine likelihood-based training with local Jacobian regularization and noise augmentation into a method that scales well to high-dimensional data such as images. Experiments on the CelebA dataset show that our method uncovers a rich set of semantically interpretable features, allowing for high compression and strong denoising.

Related papers

• ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation [9.230247128710865]
  We propose a training-free diffusion-based framework that integrates manifold consistent guidance at every denoising timestep.<n>ManifoldGD improves representativeness, diversity, and image fidelity without requiring any model retraining.
  arXiv  Detail & Related papers  (2026-02-26T18:07:10Z)
• Fast and Scalable Analytical Diffusion [43.4234198410875]
  Analytical diffusion models offer a mathematically transparent path to generative modeling by formulating the denoising score as an empirical-Bayes posterior mean.<n>Standard formulation requires a full-dataset scan at every timestep, scaling linearly with dataset size.<n>We propose Dynamic Time-Aware Golden Subset Diffusion (GoldDiff), a training-free framework that decouples inference from dataset size.
  arXiv  Detail & Related papers  (2026-02-18T14:41:09Z)
• The Entropic Signature of Class Speciation in Diffusion Models [12.212582407125089]
  We show that tracking the class-conditional entropy of a latent semantic variable given a noisy state provides a reliable signature of transition regimes.<n>We validate our method on EDM2-XS and Stable Diffusion 1.5, where class-conditional entropy consistently isolates the noise regimes critical for semantic structure formation.
  arXiv  Detail & Related papers  (2026-02-10T10:56:46Z)
• Performance Evaluation of Ising and QUBO Variable Encodings in Boltzmann Machine Learning [0.7734726150561088]
  QUBO induces larger cross terms between first- and second-order statistics, creating more small-eigenvalue directions in the Fisher information matrix.<n>Ising encoding provides more isotropic curvature and faster convergence.<n>These results clarify how representation shapes information geometry and finite-time learning dynamics in Boltzmann machines.
  arXiv  Detail & Related papers  (2025-10-15T06:57:23Z)
• Robust Canonicalization through Bootstrapped Data Re-Alignment [5.437226012505534]
  Fine-grained visual classification tasks, such as insect and bird identification, demand sensitivity to subtle visual cues.<n>We propose a bootstrapping algorithm that iteratively re-aligns training samples by reducing variance.<n>We show that our method consistently outperforms equivariant, and canonicalization baselines while performing on par with augmentation.
  arXiv  Detail & Related papers  (2025-10-09T13:05:20Z)
• Tracing the Representation Geometry of Language Models from Pretraining to Post-training [22.18942718274405]
  We take a spectral approach to investigate the geometry of learned representations across pretraining and post-training.<n>We uncover a consistent non-monotonic sequence of three geometric phases during autoregressive pretraining.<n>Post-training further transforms geometry: SFT and DPO drive "entropy-seeking" dynamics to integrate specific instructional or preferential data.
  arXiv  Detail & Related papers  (2025-09-27T00:46:29Z)
• Unfolding Generative Flows with Koopman Operators: Fast and Interpretable Sampling [25.420559502119485]
  Continuous Normalizing Flows (CNFs) enable elegant generative modeling but remain bottlenecked by slow sampling.<n>Recent approaches such as Rectified Flow and OT-CFM accelerate sampling by straightening trajectories, yet the learned dynamics remain nonlinear black boxes.<n>We propose globally linearizing flow dynamics via Koopman theory.
  arXiv  Detail & Related papers  (2025-06-27T15:16:16Z)
• Self-supervised Latent Space Optimization with Nebula Variational Coding [87.20343320266215]
  This paper proposes a variational inference model which leads to a clustered embedding.<n>We introduce additional variables in the latent space, called textbfnebula anchors, that guide the latent variables to form clusters during training.<n>Since each latent feature can be labeled with the closest anchor, we also propose to apply metric learning in a self-supervised way to make the separation between clusters more explicit.
  arXiv  Detail & Related papers  (2025-06-02T08:13:32Z)
• Adaptive Inference-Time Scaling via Cyclic Diffusion Search [61.42700671176343]
  We introduce the challenge of adaptive inference-time scaling-dynamically adjusting computational effort during inference.<n>We propose Adaptive Bi-directional Cyclic Diffusion (ABCD), a flexible, search-based inference framework.<n>ABCD refines outputs through bi-directional diffusion cycles while adaptively controlling exploration depth and termination.
  arXiv  Detail & Related papers  (2025-05-20T07:31:38Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Impact of Noisy Supervision in Foundation Model Learning [91.56591923244943]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets.<n>We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Gradient-Based Feature Learning under Structured Data [57.76552698981579]
  In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction. We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue. In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
  arXiv  Detail & Related papers  (2023-09-07T16:55:50Z)
• Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization [76.68866368409216]
  We propose learning to dynamically select discretization tightness conditioned on inputs. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.
  arXiv  Detail & Related papers  (2022-02-02T23:54:26Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)
• Focus of Attention Improves Information Transfer in Visual Features [80.22965663534556]
  This paper focuses on unsupervised learning for transferring visual information in a truly online setting. The computation of the entropy terms is carried out by a temporal process which yields online estimation of the entropy terms. In order to better structure the input probability distribution, we use a human-like focus of attention model.
  arXiv  Detail & Related papers  (2020-06-16T15:07:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.