Entropic Auto-Encoding via Implicit Free-Energy Minimization

• URL: http://arxiv.org/abs/2605.16164v1
• Date: Fri, 15 May 2026 16:43:45 GMT
• Title: Entropic Auto-Encoding via Implicit Free-Energy Minimization
• Abstract summary: We introduce Entropic Autoencoders (EAEs), a framework in which reconstruction loss is the only explicit objective.<n>We demonstrate that EAEs mitigate posterior collapse by learning non-Gaussian, multimodal latent distributions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite their ubiquity, variational autoencoders (VAEs) inherently suffer from posterior collapse, a failure mode in which latent variables are effectively ignored. This failure arises because explicit prior imposition drives optimization toward loss landscape regions corresponding to uninformative latent representations. Here, we introduce Entropic Autoencoders (EAEs), a framework in which reconstruction loss is the only explicit objective, and entropy generates the latent variables' prior implicitly through a free energy-minimizing ensemble of encoders. This ensemble biases learning toward high-volume regions of near-optimal solutions, while decoder updates direct the search trajectories toward informative latent representations. We demonstrate that EAEs mitigate posterior collapse by learning non-Gaussian, multimodal latent distributions that yield diverse, data-consistent generations and preserve different forms of underlying structure in the data. As a proof-of-concept, we show that an EAE captures a superposition of the known low-dimensional dynamics of a reaction-diffusion process. Then, we show that an EAE identifies implicit categorical distinctions in MNIST latent representations, and displays a hierarchical understanding of facial structure on the CelebA dataset, from an "all-human" face to individual-dependent features.

Related papers

• Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection [58.25418970608328]
  We study Single-Domain Generalized Object Detection (Single-DGOD), which aims to transfer a detector trained on a single source domain to multiple unseen domains.<n>Existing methods mainly rely on simulation-driven strategies, such as data augmentation or textual prompts, to enlarge the training distribution.<n>We propose Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which formulates unknown-domain generalization as a manifold regression problem.
  arXiv  Detail & Related papers  (2026-07-08T09:27:15Z)
• Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder [34.73963627819185]
  High-resolution (HR) joint optical and synthetic aperture radar (SAR) pretraining seeks modality synergy to mutually enhance single-source representations.<n>We propose CoDe-MAE, pioneering a textitbetter synergy with less alignment philosophy.<n>CoDe-MAE successfully prevents representation degradation and establishes new state-of-the-art performance across diverse single- and bi-modal downstream tasks.
  arXiv  Detail & Related papers  (2026-04-18T10:23:00Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• PH-VAE: A Polynomial Hierarchical Variational Autoencoder Towards Disentangled Representation Learning [5.478519374490149]
  The Diveral autoencoder (VAE) is a simple and efficient generative artificial intelligence method for modeling complex probability distributions of various types of data.<n>We developed a hierarchical variational autoencoder (PH-VAE), in which we used a hierarchical date format to generate or reconstruct the data distributions.<n>In doing so, we also proposed a novel Polynomial divergence in the variation loss function to replace or generalize the Kullback-Leibler (KL) variation.
  arXiv  Detail & Related papers  (2025-02-05T03:25:32Z)
• Improved Variational Inference in Discrete VAEs using Error Correcting Codes [3.053842954605396]
  This work introduces a novel method to improve inference in discrete Variational Autoencoders by reframing the problem through a generative perspective.<n>We conceptualize the model as a communication system, and propose to leverage Error-Correcting Codes (ECCs) to introduce redundancy in latent representations.<n>We present a proof-of-concept using a Discrete Variational Autoencoder with binary latent variables and low-complexity repetition codes, extending it to a hierarchical structure for disentangling global and local data features.
  arXiv  Detail & Related papers  (2024-10-10T11:59:58Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Elastic Interaction Energy-Based Generative Model: Approximation in Feature Space [14.783344918500813]
  We propose a novel approach to generative modeling using a loss function based on elastic interaction energy (EIE) The utilization of the EIE-based metric presents several advantages, including its long range property that enables consideration of global information in the distribution. Experimental results on popular datasets, such as MNIST, FashionMNIST, CIFAR-10, and CelebA, demonstrate that our EIEG GAN model can mitigate mode collapse, enhance stability, and improve model performance.
  arXiv  Detail & Related papers  (2023-03-19T03:39:31Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• The Deep Generative Decoder: MAP estimation of representations improves modeling of single-cell RNA data [0.0]
  We present a simple generative model that computes model parameters and representations directly via maximum a posteriori (MAP) estimation. The advantages of this approach are its simplicity and its capability to provide representations of much smaller dimensionality than a comparable VAE.
  arXiv  Detail & Related papers  (2021-10-13T12:17:46Z)
• Stochastic Bottleneck: Rateless Auto-Encoder for Flexible Dimensionality Reduction [16.42183040840934]
  We propose a new concept of rateless auto-encoders (RL-AEs) that enable a flexible latent dimensionality. We experimentally demonstrate that the proposed RL-AEs can achieve variable dimensionality reduction while achieving low distortion.
  arXiv  Detail & Related papers  (2020-05-06T14:47:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.