A Simplex Witness Certificate for Constant Collapse in Variational Autoencoders

• URL: http://arxiv.org/abs/2605.18224v3
• Date: Sat, 23 May 2026 10:58:41 GMT
• Title: A Simplex Witness Certificate for Constant Collapse in Variational Autoencoders
• Abstract summary: We study exact constant collapse in variational autoencoders.<n>For any full-support teacher posterior, the same geometry gives a closed-form latent code with zero teacher-witness alignment error.<n>We instantiate the method on MNIST, CIFAR-10, and CIFAR-100.
• Score: 3.812090634738739
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study exact constant collapse in variational autoencoders: the deterministic encoder mean becomes independent of the input. The prior remains the standard Gaussian. Before VAE training, we select a fixed teacher posterior from a GMM-based view of the data and attach a fixed latent-only simplex witness to the encoder mean. This construction yields two linked objects. The first is a certificate: if the witness prediction improves on the best constant predictor of the teacher, the encoder mean cannot be input-independent constant. The second is a local escape direction: on the collapsed manifold, the teacher residual gives a sample-dependent descent direction for the alignment loss. For any full-support teacher posterior, the same geometry also gives a closed-form latent code with zero teacher-witness alignment error. Its scaled versions trace a margin-energy path from the constant predictor to the exact teacher code, which quantifies non-collapse inside the protected witness subspace. We instantiate the method on MNIST, CIFAR-10, and CIFAR-100. With searched unsupervised PCA-GMM teachers, vanilla VAEs fail the teacher-witness certificate in all five seeds on CIFAR-10 and CIFAR-100, while RST variants pass in all five seeds. Under collapse-stress settings with \(β_{\mathrm{KL}}\in\{2,4,8\}\), vanilla VAE again fails in all seeds, whereas RST-alpha-prefit remains certificate-positive. Escape trajectories on both natural-image datasets increase the witness margin from a low-margin initialization and exhibit nonzero teacher-induced gradient norms. The analysis is confined to exact constant collapse of the encoder mean; generation quality, decoder use, and other collapse modes remain separate questions.

Related papers

• Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data [0.5213778368155993]
  DIFFINT is an autoencoder whose latent bottleneck is structured as a set of soft, axis-aligned interval memberships.<n>It is the only interpretable detector in the statistically-tied leading cluster of seven methods.
  arXiv  Detail & Related papers  (2026-09-03T14:05:27Z)
• CW-BASS v2: Saturation-Aware Pseudo-Label Selection for Semi-Supervised Segmentation under Foundation-Model Teachers [0.0]
  CW-BASS v2, a saturation-aware pseudo-label selection method, reads the teacher's confidence regime rather than committing to one rule.<n>It pairs held-out calibration, an unbiased per-class noise estimate, with a self-adaptive confidence floor that provably bounds retention away from 1.
  arXiv  Detail & Related papers  (2026-08-13T03:31:31Z)
• Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training [51.18425726646089]
  We propose textscSURE, a unified latent-space framework for image and video diffusion models.<n>It learns reward distributions and directly uses their reliability to guide dense post-training.<n>textscSURE-REFL achieves the highest VBench quality, semantic, and total scores among the evaluated methods.
  arXiv  Detail & Related papers  (2026-08-06T14:55:42Z)
• Conformal Orbit-Valid Trust Horizons for Equivariant World Models [6.230579198456525]
  We study trust-horizon certification for latent world models with known group symmetries.<n>Across 50 stable audits, we observe zero anti-conservative violations.
  arXiv  Detail & Related papers  (2026-06-23T06:44:27Z)
• Graph Transfer Learning via Shared Latent Geometry: Theory and Applications [12.735610064958712]
  Inference and control in engineered physical systems pay a heavy physics cost at deployment.<n>We propose an asymmetric two-pathway architecture that resolves both issues.<n>A teacher encoder consumes privileged dense states from a high-fidelity simulator and represents the system through operator-polynomial features stable under spectral perturbation.<n>A student encoder learns the same latent geometry from sparse field data and operator descriptors.
  arXiv  Detail & Related papers  (2026-05-30T13:00:13Z)
• From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents [56.31499185764872]
  Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents.<n>We propose Patches-to-Trajectories (P2T), which uses $pstar$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length.
  arXiv  Detail & Related papers  (2026-05-21T04:54:55Z)
• The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs [52.709361620508595]
  ListOPD brings a student to in-domain with an 8B-SFT baseline at one-fifth the parameters.<n>On Amazon Fashion, three pre-registered tests--a fine-grid cliff interval test, and a small-clip cross-prediction--fall within their locked prediction windows--with the small-clip value matching the closed-form prediction below grid resolution.
  arXiv  Detail & Related papers  (2026-05-09T06:48:00Z)
• A Testable Certificate for Constant Collapse in Teacher-Guided VAEs [3.812090634738739]
  Posterior collapse in variational autoencoders is often diagnosed by its symptoms.<n>We study a concrete failure mode, input-independent constant collapse, and show that this case admits an exact threshold.
  arXiv  Detail & Related papers  (2026-05-07T07:48:41Z)
• Beyond Logit Adjustment: A Residual Decomposition Framework for Long-Tailed Reranking [3.165340675929986]
  We develop a lightweight post-hoc reranker that combines a shrinkage-stabilized classwise term with a linear pairwise term driven by competition features on the shortlist.<n>Experiments on five benchmarks spanning image classification, species recognition, scene recognition, and rare disease diagnosis confirm that the decomposition explains where pairwise correction helps and where classwise correction alone suffices.
  arXiv  Detail & Related papers  (2026-04-02T00:34:12Z)
• Beyond the Loss Curve: Scaling Laws, Active Learning, and the Limits of Learning from Exact Posteriors [8.410613979416203]
  We use class-conditional normalizing flows as oracles that make exact posteriors tractable on realistic images.<n>Our framework reveals that standard metrics hide ongoing learning, mask architectural differences, and cannot diagnose the nature of distribution shift.
  arXiv  Detail & Related papers  (2026-01-30T21:08:55Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Geometric-Disentangelment Unlearning [106.99160454669902]
  gradient ascent on forget samples often harms retained knowledge.<n>We propose the Geometric-disment Unlearning (GU) that decomposes any candidate forget gradient update into tangential and normal components to retain space and executes only the normal component.<n>Our method is plug-and-play and can be attached to existing gradient-based unlearning procedures to mitigate side effects.
  arXiv  Detail & Related papers  (2025-11-21T09:58:25Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training [58.33728862521732]
  Diffusion Transformers (DiTs) deliver state-of-the-art image quality, yet their training remains notoriously slow.<n>A recent remedy -- representation alignment (REPA) that matches DiT hidden features to those of a non-generative teacher (e.g. DINO) -- dramatically accelerates the early epochs but plateaus or even degrades performance later.<n>We trace this failure to a capacity mismatch: once the generative student begins modelling the joint data distribution, the teacher's lower-dimensional embeddings and attention patterns become a straitjacket rather than a guide.<n>We introduce HASTE
  arXiv  Detail & Related papers  (2025-05-22T15:34:33Z)
• Semantic Latent Space Regression of Diffusion Autoencoders for Vertebral Fracture Grading [72.45699658852304]
  This paper proposes a novel approach to train a generative Diffusion Autoencoder model as an unsupervised feature extractor. We model fracture grading as a continuous regression, which is more reflective of the smooth progression of fractures. Importantly, the generative nature of our method allows us to visualize different grades of a given vertebra, providing interpretability and insight into the features that contribute to automated grading.
  arXiv  Detail & Related papers  (2023-03-21T17:16:01Z)
• Calibrated Teacher for Sparsely Annotated Object Detection [35.74904852812749]
  Fully supervised object detection requires training images in which all instances are annotated. This is actually impractical due to the high labor and time costs and the unavoidable missing annotations. Recent works on sparsely annotated object detection alleviate this problem by generating pseudo labels for the missing annotations. We propose a Calibrated Teacher, of which the confidence estimation of the prediction is well to match its real precision.
  arXiv  Detail & Related papers  (2023-03-14T02:02:39Z)
• Adversarial robustness of VAEs through the lens of local geometry [1.2228014485474623]
  In an unsupervised attack on variational autoencoders (VAEs), an adversary finds a small perturbation in an input sample that significantly changes its latent space encoding. This paper demonstrates that an optimal way for an adversary to attack VAEs is to exploit a directional bias of a pullback metric tensor.
  arXiv  Detail & Related papers  (2022-08-08T05:53:57Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction [49.25830718574892]
  We present a new framework named Tail-to-Tail (textbfTtT) non-autoregressive sequence prediction. Considering that most tokens are correct and can be conveyed directly from source to target, and the error positions can be estimated and corrected. Experimental results on standard datasets, especially on the variable-length datasets, demonstrate the effectiveness of TtT in terms of sentence-level Accuracy, Precision, Recall, and F1-Measure.
  arXiv  Detail & Related papers  (2021-06-03T05:56:57Z)
• Autoencoding Variational Autoencoder [56.05008520271406]
  We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. We show that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks.
  arXiv  Detail & Related papers  (2020-12-07T14:16:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.