Related papers: Deterministic Continuous Replacement: Fast and Stable Module Replacement in Pretrained Transformers

Related papers

Response Renormalization for Critical Deep Equilibrium Models [0.0]
Training through implicit differentiation requires solving an adjoint system built from the residual Jacobian.<n>We introduce Response Renormalization, a backward-pass framework that lifts selected near-pole denominators while leaving unlifted response channels unchanged.<n>We derive dense and matrix-free collective formulations, distinguish exact gradients of a modified frozen-anchor residual from backward-response surrogates, and extend the construction to Structured Implicit Layers and Vector Attractors.
arXiv Detail & Related papers (2026-08-24T18:10:30Z)
TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs [83.78708832901194]
Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild.<n>However, real-world deployment remains challenging: modalities may be missing or noisy at test time.<n>We propose a solution in which a frozen teacher guides an adaptive low-rank student via self-distillation.<n>Experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation.
arXiv Detail & Related papers (2026-08-18T23:40:28Z)
Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation [0.08057006406834467]
We introduce a classical Lindblad-inspired multi-timescale reservoir that bridges open-system dynamical principles with structured state-space modeling.<n>We evaluate the method over ten aligned seeds against standard, leaky, deep, gated, cycle, and next-generation reservoirs.<n>Across the benchmark suite, the proposed reservoir achieves the best fixed-reservoir performance on bounded NARMA-20 and the lowest mean error on Lorenz-63.
arXiv Detail & Related papers (2026-07-27T18:48:32Z)
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think [52.879387744920415]
We find that on-policy self-distillation (OPSD) consistently fails on long chain-of-thought (long-CoT) reasoning models.<n>We propose a two-step solution: first, we construct a reference-only teacher to isolate the non-transferable component of the supervision signal.<n>Second, we use pointwise mutual information (PMI) as the mechanism to transform this residual into a well-formed PMI target distribution.
arXiv Detail & Related papers (2026-07-02T14:33:07Z)
Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping [56.14767235650558]
Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length computation tasks.<n>We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops.<n>Our work suggests that "when to stop" should be treated as a training design choice, not merely an inference-time-allocation rule.
arXiv Detail & Related papers (2026-06-29T08:58:09Z)
Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Making [12.04727482907937]
CML addresses the problem of tracking the noise of primal feedback multipliers under mini-batch conditions.<n>We decompose the multiplier into an effective pressure signal for primal descent and a finite-gain multiplier.
arXiv Detail & Related papers (2026-06-05T09:35:22Z)
Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models [56.67321805551389]
Reinforcement learning (RL) has become an effective way to improve prompt alignment and perceptual quality in diffusion and flow-matching generators.<n>The sampler, controlling the exploration behavior and denoising dynamics, is thus part of the policy.<n>We propose a new sampler that balances effective exploration with stability.
arXiv Detail & Related papers (2026-05-22T11:37:22Z)
Bridging Restoration and Generation Manifolds in One-Step Diffusion for Real-World Super-Resolution [22.963799508399365]
Recent single-step distillation accelerates inference but faces a stark perception-distortion trade-off.<n>We present Adaptive Inversion and Degradation-aware Sampling for Real-ISR (IDaS-SR)<n>IDaS-SR is a one-step framework bridging the deterministic restoration and generation bottlenecks.
arXiv Detail & Related papers (2026-04-27T07:43:00Z)
Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
arXiv Detail & Related papers (2026-03-07T01:49:59Z)
CASR: A Robust Cyclic Framework for Arbitrary Large-Scale Super-Resolution with Distribution Alignment and Self-Similarity Awareness [14.01813573738112]
CASR is a simple yet highly efficient cyclic SR framework that reformulates ultra-magnification as a sequence of in-distribution scale transitions.<n>This design ensures stable inference at arbitrary scales while requiring only a single model.<n>Despite using only a single model, our approach significantly reduces distribution drift, preserves long-range texture consistency, and achieves superior generalization even at extreme magnification.
arXiv Detail & Related papers (2026-02-25T18:05:51Z)
Constraint-Rectified Training for Efficient Chain-of-Thought [60.52883907721588]
Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs)<n>While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking.<n>Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy.
arXiv Detail & Related papers (2026-02-13T02:13:45Z)
Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
arXiv Detail & Related papers (2026-02-01T12:56:10Z)
Reasoning Stabilization Point: A Training-Time Signal for Stable Evidence and Shortcut Reliance [0.0]
We define explanation drift as the epoch-to-epoch change in normalized token attributions on a fixed probe set.<n>RSP is computed from within-run drift dynamics and requires no tuning on out-of-distribution data.
arXiv Detail & Related papers (2026-01-12T17:48:05Z)
Measurement-Consistent Langevin Corrector: A Remedy for Latent Diffusion Inverse Solvers [24.18139511731393]
Latent diffusion inverse solvers suffer from instability, exhibiting undesirable artifacts and degraded quality.<n>We introduce Measurement-Consistent Langevin Corrector (MCLC), a theoretically grounded plug-and-play correction module.<n>Compared to prior approaches, MCLC operates without this assumption, leading to more stable and reliable behavior.
arXiv Detail & Related papers (2026-01-08T10:15:35Z)
GB-DQN: Gradient Boosted DQN Models for Non-stationary Reinforcement Learning [0.0]
We propose emphGradient-Boosted Deep Q-Networks (GB-DQN), an adaptive ensemble method that addresses model drift through incremental residual learning.<n>Instead of retraining a single Q-network, GB-DQN constructs an additive ensemble in which each new learner is trained to approximate the Bellman residual of the current ensemble after drift.
arXiv Detail & Related papers (2025-12-18T19:53:50Z)
Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield [54.328202401611264]
Diffusion model distillation has emerged as a powerful technique for creating efficient few-step and single-step generators.<n>We show that the primary driver of few-step distillation is not distribution matching, but a previously overlooked component we identify as CFG Augmentation (CA)<n>We propose principled modifications to the distillation process, such as decoupling the noise schedules for the engine and the regularizer, leading to further performance gains.
arXiv Detail & Related papers (2025-11-27T18:24:28Z)
CREPE: Controlling Diffusion with Replica Exchange [32.38925001748167]
Inference-time control of diffusion models aims to steer model outputs to satisfy new constraints without retraining.<n>We propose a flexible alternative based on replica exchange, an algorithm designed initially for sampling problems.<n> CREPE: (1) generates particles sequentially, (2) maintains high diversity in the generated samples after a burn-in period, and (3) enables online refinement or early termination.
arXiv Detail & Related papers (2025-09-27T11:45:37Z)
Metriplectic Conditional Flow Matching for Dissipative Dynamics [5.920407670799846]
conditional flow matching learns dissipative dynamics without violating first principles.<n>MCFM trains via conditional flow matching on short transitions, avoiding long rollout adjoints.<n>We provide continuous and discrete time guarantees linking this parameterization and sampler to conservation, monotonic dissipation, and stable rollouts.
arXiv Detail & Related papers (2025-09-23T19:46:54Z)
The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward [57.56453588632619]
A central paradox in fine-tuning Large Language Models (LLMs) with Reinforcement Learning with Verifiable Reward (RLVR) is the frequent degradation of multi-attempt performance.<n>This is often accompanied by catastrophic forgetting, where models lose previously acquired skills.<n>We argue that standard RLVR objectives lack a crucial mechanism for knowledge retention.
arXiv Detail & Related papers (2025-09-09T06:34:32Z)
Exploring Magnitude Preservation and Rotation Modulation in Diffusion Transformers [5.187307904567701]
We propose a magnitude-preserving design that stabilizes training without normalization layers.<n>Motivated by the goal of maintaining activation magnitudes, we additionally introduce rotation modulation.<n>We show that magnitude-preserving strategies significantly improve performance, notably reducing FID scores by $sim$12.8%.
arXiv Detail & Related papers (2025-05-25T12:25:50Z)
Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
We propose textitRestoration Score Distillation (RSD), a principled generalization of Denoising Score Distillation (DSD)<n>RSD accommodates a broader range of corruption types, such as blurred, incomplete, or low-resolution images.<n>It consistently surpasses its teacher model across diverse restoration tasks on both natural and scientific datasets.
arXiv Detail & Related papers (2025-05-19T17:21:03Z)
Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation [91.83820250747935]
Pseudo-label noise is mainly contained in unstable samples in which predictions of most pixels undergo significant variations during self-training. We introduce the Stable Neighbor Denoising (SND) approach, which effectively discovers highly correlated stable and unstable samples. SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings.
arXiv Detail & Related papers (2024-06-10T21:44:52Z)
Adaptive Gradient Clipping for Robust Federated Learning [8.268485501864939]
We propose a principled adaptive clipping strategy, Adaptive Robust Clipping (ARC), which dynamically adjusts clipping thresholds based on the input gradients.<n>ARC significantly enhances robustness, particularly in highly heterogeneous and adversarial settings.
arXiv Detail & Related papers (2024-05-23T11:00:31Z)
On the Convergence of Stochastic Extragradient for Bilinear Games with Restarted Iteration Averaging [96.13485146617322]
We present an analysis of the ExtraGradient (SEG) method with constant step size, and present variations of the method that yield favorable convergence. We prove that when augmented with averaging, SEG provably converges to the Nash equilibrium, and such a rate is provably accelerated by incorporating a scheduled restarting procedure.
arXiv Detail & Related papers (2021-06-30T17:51:36Z)
Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
arXiv Detail & Related papers (2020-10-28T15:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.