SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models

• URL: http://arxiv.org/abs/2604.16606v1
• Date: Fri, 17 Apr 2026 18:00:58 GMT
• Title: SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models
• Abstract summary: We present SafeLM, a framework that jointly addresses privacy, security, misinformation, and adversarial robustness.<n>SafeLM achieves 98.0% harmful content detection accuracy, reduces communication by 96.9%, and lowers PSNR from 31.7 dB to 15.1 dB.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly deployed in high-stakes domains, yet a unified treatment of their overlapping safety challenges remains lacking. We present SafeLM, a framework that jointly addresses four pillars of LLM safety: privacy, security, misinformation, and adversarial robustness. SafeLM combines federated training with gradient smartification and Paillier encryption for privacy, integrates defenses against training and inference-time attacks, employs contrastive grounding with calibrated decoding to reduce hallucinations, and introduces alignment-aware binarized aggregation to enhance robustness while maintaining bounded reconstruction quality. Across benchmarks on factuality, toxicity, and membership inference, SafeLM achieves 98.0% harmful content detection accuracy, reduces communication by 96.9%, and lowers gradient inversion PSNR from 31.7 dB to 15.1 dB. Ablations show that each component contributes independently, whereas their integration yields a strong privacy utility efficiency trade-off for deploying trustworthy LLMs.

Related papers

• aiXamine: Unified Black-Box Evaluation of Cross-Dimensional Trade-offs in LLM Safety, Security, and Privacy [6.323488051975665]
  AiXamine is a unified black-box platform that evaluates trustworthiness across safety, security, and privacy as interdependent properties.<n>AiXamine orchestrates 46 tests across nine services through an automated red-teaming pipeline.<n>We conduct the largest joint safety, security, and privacy study to date and uncover three cross-dimensional phenomena invisible to single-axis evaluation.
  arXiv  Detail & Related papers  (2026-08-20T20:33:35Z)
• Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach [44.738736515079346]
  Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data.<n>FedLLMs' unique properties, i.e. massive parameter scales, rapid convergence, and sparse, non-orthogonal gradients, render existing MIAs ineffective.<n>We propose ProjRes, the first projection residuals-based passive MIA tailored for FedLLMs.
  arXiv  Detail & Related papers  (2026-04-23T01:44:04Z)
• Robust Multimodal Safety via Conditional Decoding [52.92816441364308]
  Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions.<n>We propose a simple conditional decoding strategy, CASA, that utilizes internal representations of MLLMs to predict a binary safety token before response generation.
  arXiv  Detail & Related papers  (2026-03-31T23:19:50Z)
• Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions [89.52532304099522]
  Federated Learning (FL) enables clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy.<n> gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation.<n>These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework.<n>We propose Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping.
  arXiv  Detail & Related papers  (2026-03-24T17:39:09Z)
• Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Framework [0.6015898117103067]
  Large Language Models increasingly power critical infrastructure from healthcare to finance, yet their vulnerability to adversarial manipulation threatens system integrity and user safety.<n>This research addresses this gap by establishing a standardized vulnerability assessment framework and developing a multi-layered defensive system to protect against identified threats.
  arXiv  Detail & Related papers  (2026-03-17T20:32:06Z)
• Coding-Enforced Resilient and Secure Aggregation for Hierarchical Federated Learning [30.254515308020512]
  Hierarchical federated learning (HFL) has emerged as an effective paradigm to enhance link quality between clients and the server.<n>We propose a robust hierarchical secure aggregation scheme, termed H-SecCoGC, which integrates coding strategies to enforce structured aggregation.
  arXiv  Detail & Related papers  (2026-01-25T21:07:22Z)
• FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Health [0.0]
  FedMentor is a fine-tuning framework for privacy-preserving adaptation of Large Language Models in sensitive domains.<n>It integrates Low-Rank Adaptation (LoRA) and domain-aware Differential Privacy (DP) to meet per-domain privacy budgets while maintaining performance.<n>In experiments on three mental health datasets, we show that FedMentor improves safety over standard Federated Learning (FL) without privacy.
  arXiv  Detail & Related papers  (2025-09-16T07:08:36Z)
• Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models [43.88239953205896]
  We introduce Omni-SafetyBench, the first comprehensive parallel benchmark for OLLM safety evaluation.<n>Considering OLLMs' comprehension challenges with complex omni-modal inputs, we propose a Safety-score based on Conditional Attack Success Rate (C-ASR) and Refusal Rate (C-RR) to account for comprehension failures.<n>Using Omni-SafetyBench, we evaluated existing safety alignment algorithms and identified key challenges in OLLM safety alignment.
  arXiv  Detail & Related papers  (2025-08-10T04:15:16Z)
• Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission [87.68447072141402]
  Hybrid Language Models (HLMs) combine the low-latency efficiency of Small Language Models (SLMs) on edge devices with the high accuracy of Large Language Models (LLMs) on centralized servers.<n>We propose FedHLM, a communication-efficient HLM framework that integrates uncertainty-aware inference with Federated Learning (FL)
  arXiv  Detail & Related papers  (2025-06-30T02:56:11Z)
• Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Models [50.89022445197919]
  Large Audio Language Models (LALMs) have extended the capabilities of Large Language Models (LLMs)<n>Recent research has revealed that LALMs remain vulnerable to harmful queries due to insufficient safety-alignment.
  arXiv  Detail & Related papers  (2025-05-26T08:25:25Z)
• PriFFT: Privacy-preserving Federated Fine-tuning of Large Language Models via Hybrid Secret Sharing [20.148411915688175]
  Fine-tuning large language models (LLMs) raises privacy concerns due to the risk of exposing sensitive training data.<n>Recent studies show that adversaries can still infer private information in FL.<n>We propose PriFFT, a privacy-preserving federated fine-tuning mechanism.
  arXiv  Detail & Related papers  (2025-03-05T03:41:57Z)
• CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration [90.36429361299807]
  multimodal large language models (MLLMs) have demonstrated remarkable success in engaging in conversations involving visual inputs. The integration of visual modality has introduced a unique vulnerability: the MLLM becomes susceptible to malicious visual inputs. We introduce a technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution.
  arXiv  Detail & Related papers  (2024-09-17T17:14:41Z)
• Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching [74.62818936088065]
  textscSafePatching is a novel framework for comprehensive PSA.<n>textscSafePatching achieves a more comprehensive PSA than baseline methods.<n>textscSafePatching demonstrates its superiority in continual PSA scenarios.
  arXiv  Detail & Related papers  (2024-05-22T16:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.