The Devil Is in Gradient Entanglement: Energy-Aware Gradient Coordinator for Robust Generalized Category Discovery

• URL: http://arxiv.org/abs/2604.14176v1
• Date: Wed, 25 Mar 2026 14:58:42 GMT
• Title: The Devil Is in Gradient Entanglement: Energy-Aware Gradient Coordinator for Robust Generalized Category Discovery
• Abstract summary: Energy-Aware Gradient Coordinator (EAGC) is a plug-and-play gradient module that explicitly regulates the optimization process.<n>EAGC comprises two components: Anchor-based Gradient Alignment (AGA) and Energy-aware Elastic Projection (EEP)
• Score: 76.35318747478522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized Category Discovery (GCD) leverages labeled data to categorize unlabeled samples from known or unknown classes. Most previous methods jointly optimize supervised and unsupervised objectives and achieve promising results. However, inherent optimization interference still limits their ability to improve further. Through quantitative analysis, we identify a key issue, i.e., gradient entanglement, which 1) distorts supervised gradients and weakens discrimination among known classes, and 2) induces representation-subspace overlap between known and novel classes, reducing the separability of novel categories. To address this issue, we propose the Energy-Aware Gradient Coordinator (EAGC), a plug-and-play gradient-level module that explicitly regulates the optimization process. EAGC comprises two components: Anchor-based Gradient Alignment (AGA) and Energy-aware Elastic Projection (EEP). AGA introduces a reference model to anchor the gradient directions of labeled samples, preserving the discriminative structure of known classes against the interference of unlabeled gradients. EEP softly projects unlabeled gradients onto the complement of the known-class subspace and derives an energy-based coefficient to adaptively scale the projection for each unlabeled sample according to its degree of alignment with the known subspace, thereby reducing subspace overlap without suppressing unlabeled samples that likely belong to known classes. Experiments show that EAGC consistently boosts existing methods and establishes new state-of-the-art results. Code is available at https://haiyangzheng.github.io/EAGC.

Related papers

• Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning [15.386002863432678]
  Open-set semi-supervised learning aims to leverage unlabeled data that may contain out-of-distribution outliers.<n>Existing methods mainly follow two paradigms: filtering suspicious samples or incorporating unlabeled objectives with soft weighting.<n>We argue that both face a common trade-off: aggressive filtering can discard informative but hard ID samples, whereas utilization can introduce auxiliary gradients that conflict with supervised learning when pseudo labels are wrong.<n>We propose textitGeometric Gradient Rectification (GGR), a plug-in framework that uses the supervised gradient as an anchor and projects conflicting auxiliary gradients onto an admissible
  arXiv  Detail & Related papers  (2026-06-25T12:45:42Z)
• Gradient Perturbation: Learning to Perturb Gradients for Adaptive Training [3.2608238154676843]
  gradient perturbation has received little systematic investigation.<n>We propose Learning to Perturb Gradients (LPG), which adaptively perturbs logit-level gradients at the class level to achieve category-aware training.<n> Experiments on balanced classification, long-tail classification, and noisy label learning demonstrate LPG consistently outperforms existing methods and can be combined with them as a plug-in module.
  arXiv  Detail & Related papers  (2026-05-28T07:19:36Z)
• SURGE: Surrogate Gradient Adaptation in Binary Neural Networks [41.424349870612716]
  We propose SURrogate GradiEnt Adaptation (SURGE), a learnable gradient compensation framework with theoretical grounding.<n> SURGE mitigates gradient mismatch through auxiliary backpropagation.<n> Experiments on image classification, object detection, and language understanding tasks demonstrate that SURGE performs best over state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-09T09:52:38Z)
• Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning [7.794885131732119]
  We show that flatness and gradient alignment should be considered in multi-distribution learning settings.<n>We propose SAGE (Spectral-Aware Gradient-Aligned Exploration) that targets both terms.<n> Experiments on five domain-generalization and two multi-task learning benchmarks show that the proposed method establishes a new state-of-the-art.
  arXiv  Detail & Related papers  (2026-05-08T15:52:34Z)
• Grad-ELLM: Gradient-based Explanations for Decoder-only LLMs [52.15785423211181]
  Grad-ELLM is a gradient-based attribution method for decoder-only transformer-based Large Language Models.<n>We introduce two faithfulneses metrics $$-Soft-NC and $$-Soft-NS, which provide fairer comparisons.<n>Experiment results show that Grad-ELLM consistently achieves superior faithfulness than other attribution methods.
  arXiv  Detail & Related papers  (2026-01-06T15:22:39Z)
• A New Formulation of Lipschitz Constrained With Functional Gradient Learning for GANs [52.55025869932486]
  This paper introduces a promising alternative method for training Generative Adversarial Networks (GANs) on large-scale datasets with clear theoretical guarantees.<n>We propose a novel Lipschitz-constrained Functional Gradient GANs learning (Li-CFG) method to stabilize the training of GAN.<n>We demonstrate that the neighborhood size of the latent vector can be reduced by increasing the norm of the discriminator gradient.
  arXiv  Detail & Related papers  (2025-01-20T02:48:07Z)
• Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradients [11.269920973751244]
  In personalized Federated Learning (pFL), high data heterogeneity can cause significant gradient divergence across devices. We introduce a new approach to pFL design, namely Federated Learning with Layer-wise Aggregation via Gradient Analysis (FedLAG) FedLAG assigns layers for personalization based on the extent of layer-wise gradient conflicts.
  arXiv  Detail & Related papers  (2024-10-03T14:46:19Z)
• LayerMatch: Do Pseudo-labels Benefit All Layers? [77.59625180366115]
  Semi-supervised learning offers a promising solution to mitigate the dependency of labeled data. We develop two layer-specific pseudo-label strategies, termed Grad-ReLU and Avg-Clustering. Our approach consistently demonstrates exceptional performance on standard semi-supervised learning benchmarks.
  arXiv  Detail & Related papers  (2024-06-20T11:25:50Z)
• Class Gradient Projection For Continual Learning [99.105266615448]
  Catastrophic forgetting is one of the most critical challenges in Continual Learning (CL) We propose Class Gradient Projection (CGP), which calculates the gradient subspace from individual classes rather than tasks.
  arXiv  Detail & Related papers  (2023-11-25T02:45:56Z)
• Mind the Gap: Polishing Pseudo labels for Accurate Semi-supervised Object Detection [18.274860417877093]
  We propose a dual pseudo-label polishing framework for semi-supervised object detection (SSOD) Instead of directly exploiting the pseudo labels produced by the teacher detector, we take the first attempt at reducing their deviation from ground truth. By doing this, both polishing networks can infer more accurate pseudo labels for unannotated objects.
  arXiv  Detail & Related papers  (2022-07-17T14:07:49Z)
• Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels [92.98756432746482]
  We study a weakly supervised problem called learning with complementary labels. We show that the quality of gradient estimation matters more in risk minimization. We propose a novel surrogate complementary loss(SCL) framework that trades zero bias with reduced variance.
  arXiv  Detail & Related papers  (2020-07-05T04:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.