OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization

• URL: http://arxiv.org/abs/2605.20276v1
• Date: Tue, 19 May 2026 04:13:27 GMT
• Title: OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization
• Abstract summary: We propose OmniISR, a unified framework that fuses pure CL, pure FL, and hybrid CL-FL training modes.<n>We show that OmniISR consistently improves model performance in both centralized and federated paradigms.
• Score: 58.03221830946145
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The global deployment of edge intelligence operates across heterogeneous legal frameworks. While some regions permit centralized learning (CL) via cloud data aggregation, others enforce strict data localization, necessitating federated learning (FL). This operational dichotomy introduces two incompatible optimization regimes (i.e., unbiased global gradients yet coupled with internal covariate shift in CL versus biased, drift-prone local updates in FL), resulting in that any naive integration of the two lacks rigorous theoretical guarantees. To fill this gap, we propose OmniISR, a unified framework that fuses pure CL, pure FL, and hybrid CL-FL training modes via equipping intermediate supervision and regularization (ISR) signals at multiple hidden layers. Specifically, we propose (i) to use mutual-information (MI) as intermediate supervision to align shifting internal covariate in CL and client-drifting representations in FL, and (ii) to adopt negative-entropy (NE) as intermediate regularizer to penalize overconfident prediction, preserve representational uncertainty, and avoid device-specific collapse. On the theory side, we derive (i) a unified, ISR-agnostic, and non-asymptotic O(1/sqrt(T)) convergence bound that shows the introduced ISR does not violate standard SGD convergence, (ii) a federated drift-bound that quantifies the ISR-reduced client drift, (iii) a gradient-alignment guarantee that ensures non-conflicting CL and FL updates under mild bias, and (iv) an explicit escape-time bound that indicates that CL-FL hybrid mixing enlarges effective stochasticity and accelerates escape from strict saddles. Extensive experiments demonstrate that OmniISR consistently improves model performance in both centralized and federated paradigms, reduces the CL-FL gap by 22.60%, and yields 37/48 paired metric wins across multiple FL algorithms.

Related papers

• Proportional-Fair Resource Allocation and Dual-Threshold Early-Exit Inference for Secure Cooperative Multi-Layer Edge Intelligence [9.686757288356816]
  This paper proposes a secure wireless edge-intelligence framework for event-triggered inference in a cooperative user equipment (UE)--edge server (ES)--cloud system.<n>We formulate a proportionally-fair utility problem that jointly optimize UE--ES association, wireless and processing resources, and confidence thresholds.<n> Numerical results with early-exit MobileNetV2 and ShuffleNetV2 demonstrate near-perfect fairness with aggregate utility close to a Sum-Utility benchmark, reveal security-induced resource fragmentation, and demonstrate Stage-A scalability from 6 to 144 UEs with median solving time below 0.1s in the tested configurations.
  arXiv  Detail & Related papers  (2026-09-14T16:41:15Z)
• Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem [12.498691027665332]
  Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems.<n>FCL requires models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift.<n>We propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lynov drift-plus-penalty (DPP) optimization.
  arXiv  Detail & Related papers  (2026-08-21T18:21:19Z)
• Towards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers [49.91939413642465]
  We investigate cluster formation, involving number and composition of clusters, in decentralized federated learning (FL)<n>We propose serverless semi-decentralized FL (SSD-FL), a methodology requiring no persistent server infrastructure.<n>SSD-FL segments global rounds into intra-cluster and inter-cluster regimes, ensuring global convergence and consensus.
  arXiv  Detail & Related papers  (2026-06-04T20:05:18Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning [52.83811558753284]
  Decentralized learning (DFL) discards the central server and establishes a decentralized communication network. Existing DFL methods still suffer from two major challenges: local inconsistency and local overfitting.
  arXiv  Detail & Related papers  (2023-08-16T11:22:36Z)
• Clustered Federated Learning based on Nonconvex Pairwise Fusion [22.82565500426576]
  We introduce a novel clustered FL setting called Fusion Clustering (FPFC) FPFC can perform partial updates at each communication allows parallel computation with variable workload. We also propose a new practical strategy for FLFC with general losses and robustness.
  arXiv  Detail & Related papers  (2022-11-08T13:04:56Z)
• Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring [104.19414150171472]
  Attributes skews the current federated learning (FL) frameworks from consistent optimization directions among the clients. We propose disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches. Experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods.
  arXiv  Detail & Related papers  (2022-06-14T13:12:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.