A Mechanism-Driven Theory of Phase Transitions in Active Learning

• URL: http://arxiv.org/abs/2607.00144v1
• Date: Tue, 30 Jun 2026 20:20:33 GMT
• Title: A Mechanism-Driven Theory of Phase Transitions in Active Learning
• Abstract summary: Active learning (AL) performance is known to be budget-dependent, yet regimes fail to generalize across datasets or architectures.<n>We characterize AL dynamics by reframing budget regimes as shifts in the dominant generalization mechanism.<n>This work provides a unified framework for the next generation of transition-aware AL algorithms.
• Score: 3.5819392925290043
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Active learning (AL) performance is known to be budget-dependent, yet regimes are typically defined by heuristic label counts that fail to generalize across datasets or architectures. We characterize AL dynamics by reframing budget regimes as shifts in the dominant generalization mechanism. By reinterpreting PAC-style risk components as dynamic interacting terms, we prove that dominance shifts are structurally unavoidable, creating a moving bottleneck for generalization. We operationalize this using measurable proxies and a segmented regression procedure to identify a tripartite taxonomy: data-driven, transition, and model-driven phases. Our framework explains the long-standing observation that representativeness, coverage, and uncertainty strategies excel at different stages. Experiments across natural and medical imaging show that AL efficiency depends on the alignment between the strategy's inductive bias and the active bottleneck. Moreover, self-supervised representation shift transitions earlier along the labeling trajectory, highlighting the role of representation quality in shaping AL dynamics. Overall, this work provides a unified framework for the next generation of transition-aware AL algorithms.

Related papers

• Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models [6.732084106610505]
  World Action Models (WAMs) augment robot policies by predicting how task-relevant scene states may evolve under interaction.<n>Latent transitions are commonly realized with Transformer-based predictors whose inductive structure is centered on token interaction rather than temporal evolution.<n>We introduce the Latent Evolution Operator Network (LEON), which models latent evolution in a learned observable space through context-modulated operator-based propagation and additive forcing.
  arXiv  Detail & Related papers  (2026-08-27T15:43:44Z)
• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models [22.16380408116294]
  World Action Models (WAMs) aim to construct a unified architecture capable of understanding world state evolution and guiding to generative motion planning.<n>We propose PILOT, whose core Representational Deduction (RD) bridges this gap by integrating motion thought-of-chain guidance as a native model capability.
  arXiv  Detail & Related papers  (2026-08-07T09:12:15Z)
• Online Continual Learning with Dynamic Label Hierarchies [72.33335166136045]
  We introduce a new problem setting, DHOCL, where severity evolve across granularities and each sample provides supervision at a single hierarchical level.<n>In this setting, we find two fundamental issues: (i) partial supervision under mixed granularities provides only point-wise signals over an evolving path-wise hierarchy, which constrains plasticity and undermines cross-level semantic consistency.<n>We propose HALO, which adaptively combines complementary classification heads, regularized by organized learnable hierarchical prototypes, enabling rapid adaptation, hierarchical consistency, and structured knowledge consolidation as the taxonomy evolves.
  arXiv  Detail & Related papers  (2026-05-12T08:20:23Z)
• Structural Instability of Feature Composition [2.5754366051855837]
  We study the compositional-collapse threshold under a spherical dictionary model.<n>We show that, in the high-bias regime, ReLU converts microscopic correlation-induced variance fluctuations into a systematic drift.<n>We validate the predicted scaling trends on structured semantic features extracted from CLEVR.
  arXiv  Detail & Related papers  (2026-04-18T05:53:31Z)
• Learning in Markov Decision Processes with Exogenous Dynamics [39.6376520918509]
  We study a structured class of MDPs characterized by state components whose transitions are independent of the agent's actions.<n>We show that exploiting this structure yields significantly improved learning guarantees.<n>We empirically validate our approach across classical toy settings and real-world-inspired environments.
  arXiv  Detail & Related papers  (2026-03-03T11:10:45Z)
• Graded Transformers [0.0]
  We introduce the Graded Transformer framework, a new class of sequence models that embed inductive biases through grading on vector spaces.<n>The framework enables adaptive feature prioritization, overcoming limitations of fixed grades in earlier models.<n>The Graded Transformer provides a mathematically principled approach to hierarchical learning and neuro-symbolic reasoning.
  arXiv  Detail & Related papers  (2025-07-27T02:34:08Z)
• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• Situationally-Aware Dynamics Learning [57.698553219660376]
  We propose a novel framework for online learning of hidden state representations.<n>Our approach explicitly models the influence of unobserved parameters on both transition dynamics and reward structures.<n>Experiments in both simulation and real world reveal significant improvements in data efficiency, policy performance, and the emergence of safer, adaptive navigation strategies.
  arXiv  Detail & Related papers  (2025-05-26T06:40:11Z)
• TRACE for Tracking the Emergence of Semantic Representations in Transformers [10.777646083061395]
  We introduce TRACE, a diagnostic framework combining geometric, informational, and linguistic signals to detect phase transitions in Transformer-based LMs.<n>Experiments reveal that phase transitions align with clear intersections between curvature collapse and dimension stabilisation; these geometric shifts coincide with emerging syntactic and semantic accuracy.<n>This work advances our understanding of how linguistic abstractions emerge in LMs, offering insights into model interpretability, training efficiency, and compositional generalisation.
  arXiv  Detail & Related papers  (2025-05-23T15:03:51Z)
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective [72.55093886515824]
  We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables. We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
  arXiv  Detail & Related papers  (2021-11-29T18:59:09Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.