A Testable Certificate for Constant Collapse in Teacher-Guided VAEs

• URL: http://arxiv.org/abs/2605.05813v1
• Date: Thu, 07 May 2026 07:48:41 GMT
• Title: A Testable Certificate for Constant Collapse in Teacher-Guided VAEs
• Abstract summary: Posterior collapse in variational autoencoders is often diagnosed by its symptoms.<n>We study a concrete failure mode, input-independent constant collapse, and show that this case admits an exact threshold.
• Score: 3.812090634738739
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Posterior collapse in variational autoencoders is often diagnosed by its symptoms: a small KL term, a strong decoder, or weak use of the latent code. These signals are useful, but they do not define a collapse boundary. We study a concrete failure mode, input-independent constant collapse, and show that this case admits an exact threshold. For any fixed nonconstant teacher distribution \(T(\cdot\mid x)\), the best constant student is the dataset-average teacher distribution, and its alignment cost is the teacher mutual information \(I_T(X;T)\). Therefore, if a strictly latent-only raw witness achieves alignment loss below this value, with a safety margin, the witness cannot be constant in the input. This identity turns a qualitative failure mode into a measurable one. In CIFAR-100 experiments with per-seed teacher search, full training stays on the certified side of the boundary, removing alignment drives the raw witness into the constant-student regime, and restarting from a collapsed checkpoint with alignment enabled restores the certificate. Tiny-ImageNet-200 fixed-target runs show the same prevention--collapse--rescue pattern across three independently searched teachers. Standard VAE-style baselines, including methods that preserve reconstruction quality or post-hoc predictability, remain negative under the raw certificate. The guarantee is intentionally narrow: it certifies that the matched nonconstant teacher-relative variation passes through the latent pathway, rather than claiming that all forms of posterior collapse have been ruled out.

Related papers

• A distribution-free certification framework for trustworthy crash-severity prediction [1.089614199781423]
  We develop a certification layer that wraps any severity model unmodified.<n>On 5.2 million Texas records across seven base models spanning four decades, the layer attaches identical validity.<n>The framework is released as an open-source package with theorem-level tests.
  arXiv  Detail & Related papers  (2026-09-10T14:20:08Z)
• When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agents [8.579680185421324]
  We argue that update admission must be assessed through both error control and retained learning opportunities.<n>A standard paired-binomial construction reduces this burden when outcome disagreements are rare.<n>A separate learned-dynamics stress test distinguishes model bias from feedback-selection error.
  arXiv  Detail & Related papers  (2026-09-09T22:25:16Z)
• Beyond Polarization: The Generative Constraint of Chain-of-Thought in Pointwise Reranking [28.78994214839467]
  In pointwise document reranking, Chain-of-Thought models typically underperform direct scoring models.<n>Our empirical study first confirms that this gap is stable across scales up to 32B parameters, ruling out model and data capacity confounders.<n>We then apply stress tests utilizing reinforcement learning, fine-grained supervision, and architectural decoupling to explicitly repair these deviations.
  arXiv  Detail & Related papers  (2026-08-31T07:51:49Z)
• H$^2$SD: Hybrid Hindsight Self-Distillation [61.71131241473028]
  Reinforcement learning with verifiable rewards (RLVR) provides reliable outcome supervision for language model reasoning.<n>Existing self-distillation methods add a privileged teacher but typically assign it a fixed role.<n>We introduce Hybrid Hindsight Self-Distillation ($mathrmH2mathrmSD$), which jointly adapts teacher context and update strategy to trajectory correctness.
  arXiv  Detail & Related papers  (2026-07-21T10:47:27Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection [56.362776482614976]
  Within-dataset class-split evaluation is widely used as a proxy for fully unconditional out-of-distribution anomaly detection.<n>We show that this protocol can become ill-posed when the held-out anomaly class overlaps the normal mixture in representation space.<n>We introduce a simple training-free diagnostic, neighborhood class leakage, and show that it predicts score-direction instability across Fashion-MNIST, CIFAR-10, and Imagenette.
  arXiv  Detail & Related papers  (2026-05-23T06:27:42Z)
• A Simplex Witness Certificate for Constant Collapse in Variational Autoencoders [3.812090634738739]
  We study exact constant collapse in variational autoencoders.<n>For any full-support teacher posterior, the same geometry gives a closed-form latent code with zero teacher-witness alignment error.<n>We instantiate the method on MNIST, CIFAR-10, and CIFAR-100.
  arXiv  Detail & Related papers  (2026-05-18T11:09:22Z)
• Distill-Belief: Closed-Loop Inverse Source Localization and Characterization in Physical Fields [19.32007397161956]
  We propose textbfDistill-Belief, a teacher-student framework that decouples correctness from efficiency.<n>Experiments show that Distill-Belief consistently reduces sensing cost and improves success, posterior contraction, and estimation accuracy over baselines.
  arXiv  Detail & Related papers  (2026-04-28T20:18:29Z)
• Continual Calibration: Coverage Can Collapse Before Accuracy in Lifelong LLM Fine-Tuning [6.908972852063454]
  Uncertainty reliability can degrade earlier and more sharply than top-1 performance.<n>We study this empirically by measuring conformal coverage and calibration error on sequentially fine-tuned models.<n>We propose calibration replay, a lightweight post-hoc procedure that maintains a task-specific buffer and refits a task-specific conformal threshold.
  arXiv  Detail & Related papers  (2026-04-27T03:03:38Z)
• The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillation [67.26315138466312]
  We propose a calibration-aware OPD framework, CaOPD, that estimates empirical confidence from model rollouts.<n>Our findings highlight that capability distillation does not imply confidence, and that confidence should be treated as an essential objective in post-training.
  arXiv  Detail & Related papers  (2026-04-18T04:43:40Z)
• LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs [61.06744611795341]
  Medical vision-language models (VLMs) are strong zero-shot recognizers for medical imaging.<n>We propose texttttextbfLATA (Laplacian-Assisted Transductive Adaptation), a textittraining- and label-free refinement.<n>texttttextbfLATA sharpens zero-shot predictions without compromising exchangeability.
  arXiv  Detail & Related papers  (2026-02-19T16:45:38Z)
• MathLedger: A Verifiable Learning Substrate with Ledger-Attested Feedback [0.0]
  Contemporary AI systems achieve extraordinary performance yet remain opaque and non-verifiable.<n>We introduce MathLedger, a substrate for verifiable machine cognition that integrates formal verification, cryptographic attestation, and learning dynamics.<n>The contribution is infrastructural: a working prototype of ledger-attested learning that enables auditability at scale.
  arXiv  Detail & Related papers  (2025-12-22T19:27:55Z)
• Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing [87.48628403354351]
  certification for machine learning is proving that no adversarial sample can evade a model within a range under certain conditions. Common certification methods for segmentation use a flat set of fine-grained classes, leading to high abstain rates due to model uncertainty. We propose a novel, more practical setting, which certifies pixels within a multi-level hierarchy, and adaptively relaxes the certification to a coarser level for unstable components.
  arXiv  Detail & Related papers  (2024-02-13T11:59:43Z)
• Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
  We propose a novel method dubbed ShrinkMatch to learn uncertain samples. For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class. We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
  arXiv  Detail & Related papers  (2023-08-13T14:05:24Z)
• Semantic Latent Space Regression of Diffusion Autoencoders for Vertebral Fracture Grading [72.45699658852304]
  This paper proposes a novel approach to train a generative Diffusion Autoencoder model as an unsupervised feature extractor. We model fracture grading as a continuous regression, which is more reflective of the smooth progression of fractures. Importantly, the generative nature of our method allows us to visualize different grades of a given vertebra, providing interpretability and insight into the features that contribute to automated grading.
  arXiv  Detail & Related papers  (2023-03-21T17:16:01Z)
• Robust Mean Teacher for Continual and Gradual Test-Time Adaptation [5.744133015573047]
  Gradual test-time adaptation (TTA) considers not only a single domain shift, but a sequence of shifts. We propose and show that in the setting of TTA, the symmetric cross-entropy is better suited as a consistency loss for mean teachers. We demonstrate the effectiveness of our proposed method 'robust mean teacher' (RMT) on the continual and gradual corruption benchmarks.
  arXiv  Detail & Related papers  (2022-11-23T16:14:45Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.