Revisiting ML Training under Fully Homomorphic Encryption: Convergence Guarantees, Differential Privacy, and Efficient Algorithms

• URL: http://arxiv.org/abs/2605.27782v1
• Date: Wed, 27 May 2026 00:09:01 GMT
• Title: Revisiting ML Training under Fully Homomorphic Encryption: Convergence Guarantees, Differential Privacy, and Efficient Algorithms
• Abstract summary: We present the first theoretical convergence analysis of machine learning training under fully homomorphic encryption (FHE)<n>We show convergence of approximate gradient descent using approximations of activation and loss functions, which are required for FHE compatibility.<n>We also provide data-independent selection theoretically grounded strategies for scalable approximation which can be of independent interest.
• Score: 20.704763006949197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the first theoretical convergence analysis of machine learning training under fully homomorphic encryption (FHE), combined with a differentially private (DP) training algorithm tailored to encrypted computation. Our approach improves computational efficiency over standard differentially private gradient descent (DP-GD) while achieving comparable utility. In particular, we prove convergence of approximate gradient descent using polynomial approximations of activation and loss functions, which are required for FHE compatibility. To preserve privacy in downstream tasks, we integrate differential privacy without relying on costly per-sample gradient clipping, enabling scalable encrypted learning. We also provide data-independent hyperparameter selection and theoretically grounded strategies for polynomial approximation which can be of independent interest. Together, these contributions advance the feasibility of efficient, private, and secure machine learning on sensitive data.

Related papers

• Kernel-Based ReLU Approximation for Homomorphic Encryption-Compatible Privacy-preserving Deep Learning Models [1.392661575766483]
  Homomorphic Encryption (HE) offers a way to perform computations on encrypted data without decryption during operations.<n>HE is limited to addition and multiplication, making non-linear functions incompatible in their original form.<n>This paper proposes a kernel-based approximation of ReLU, enabling its use within HE-constrained settings.
  arXiv  Detail & Related papers  (2026-05-22T13:54:37Z)
• Linear-Time User-Level DP-SCO via Robust Statistics [55.350093142673316]
  User-level differentially private convex optimization (DP-SCO) has garnered significant attention due to the importance of safeguarding user privacy in machine learning applications.<n>Current methods, such as those based on differentially private gradient descent (DP-SGD), often struggle with high noise accumulation and suboptimal utility.<n>We introduce a novel linear-time algorithm that leverages robust statistics, specifically the median and trimmed mean, to overcome these challenges.
  arXiv  Detail & Related papers  (2025-02-13T02:05:45Z)
• HERL: Tiered Federated Learning with Adaptive Homomorphic Encryption using Reinforcement Learning [12.628921853388862]
  HERL is a Reinforcement Learning-based approach that uses Q-Learning to dynamically optimize encryption parameters. Our proposed method involves first profiling and tiering clients according to the chosen clustering approach. Our results show that HERL improves utility by 17%, reduces the convergence time by up to 24%, and increases efficiency by up to 30%, with minimal security loss.
  arXiv  Detail & Related papers  (2024-09-11T21:26:23Z)
• Privacy-aware Berrut Approximated Coded Computing for Federated Learning [1.2084539012992408]
  We propose a solution to guarantee privacy in Federated Learning schemes.<n>Our proposal is based on the Berrut Approximated Coded Computing, adapted to a Secret Sharing configuration.
  arXiv  Detail & Related papers  (2024-05-02T20:03:13Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• When approximate design for fast homomorphic computation provides differential privacy guarantees [0.08399688944263842]
  Differential privacy (DP) and cryptographic primitives are popular countermeasures against privacy attacks. In this paper, we design SHIELD, a probabilistic approximation algorithm for the argmax operator. Even if SHIELD could have other applications, we here focus on one setting and seamlessly integrate it in the SPEED collaborative training framework.
  arXiv  Detail & Related papers  (2023-04-06T09:38:01Z)
• Differentially Private Stochastic Gradient Descent with Low-Noise [49.981789906200035]
  Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. This paper addresses the practical and theoretical importance of developing privacy-preserving machine learning algorithms that ensure good performance while preserving privacy.
  arXiv  Detail & Related papers  (2022-09-09T08:54:13Z)
• Decentralized Stochastic Optimization with Inherent Privacy Protection [103.62463469366557]
  Decentralized optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data, privacy protection has become an increasingly pressing need in the implementation of decentralized optimization algorithms.
  arXiv  Detail & Related papers  (2022-05-08T14:38:23Z)
• Privacy Analysis of Online Learning Algorithms via Contraction Coefficients [5.333582981327498]
  We show that differential privacy guarantees can be determined by a direct application of contraction coefficients derived from strong data processing inequalities for $f$-divergences. We also show that this framework can be tailored to batch learning algorithms that can be implemented with one pass over the training dataset.
  arXiv  Detail & Related papers  (2020-12-20T22:02:15Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization [77.43075255745389]
  Federated learning (FL) is capable of preserving private data from mobile terminals (MTs) while training the data into useful models. From a viewpoint of information theory, it is still possible for a curious server to infer private information from the shared models uploaded by MTs. We propose a user-level differential privacy (UDP) algorithm by adding artificial noise to the shared models before uploading them to servers.
  arXiv  Detail & Related papers  (2020-02-29T10:13:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.