DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacy

• URL: http://arxiv.org/abs/2604.15851v1
• Date: Fri, 17 Apr 2026 09:03:11 GMT
• Title: DPrivBench: Benchmarking LLMs' Reasoning for Differential Privacy
• Abstract summary: Differential privacy (DP) has a wide range of applications for protecting data privacy, but designing and verifying DP algorithms requires expert-level reasoning.<n>We introduce DPrivBench, a benchmark in which each instance asks whether a function or algorithm satisfies a stated DP guarantee.<n>Experiments show that while the strongest models handle textbook mechanisms well, all models struggle with advanced algorithms.
• Score: 50.73304874527716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differential privacy (DP) has a wide range of applications for protecting data privacy, but designing and verifying DP algorithms requires expert-level reasoning, creating a high barrier for non-expert practitioners. Prior works either rely on specialized verification languages that demand substantial domain expertise or remain semi-automated and require human-in-the-loop guidance. In this work, we investigate whether large language models (LLMs) can automate DP reasoning. We introduce DPrivBench, a benchmark in which each instance asks whether a function or algorithm satisfies a stated DP guarantee under specified assumptions. The benchmark is carefully designed to cover a broad range of DP topics, span diverse difficulty levels, and resist shortcut reasoning through trivial pattern matching. Experiments show that while the strongest models handle textbook mechanisms well, all models struggle with advanced algorithms, revealing substantial gaps in current DP reasoning capabilities. Through further analytic study and failure-mode analysis, we identify several promising directions for improving automated DP reasoning. Our benchmark provides a solid foundation for developing and evaluating such methods, and complements existing benchmarks for mathematical reasoning.

Related papers

• Multi-tier Differential Private Query Release [57.049084914202275]
  Multiple analysts may issue the same query, while receiving answers generated under different privacy budgets.<n>Existing approaches for such multi-tier DP queries either incur excessive cumulative privacy loss or suffer from suboptimal utility.<n>We propose a framework for multi-tier DP query release that simultaneously bound the cumulative privacy loss by the maximum privacy budget among all queries.
  arXiv  Detail & Related papers  (2026-06-14T02:10:22Z)
• Differentially Private Retrieval-Augmented Generation [13.622078883013442]
  Retrieval-augmented generation (RAG) is a widely used framework for reducing hallucinations in large language models (LLMs)<n>RAG poses serious privacy risks when the database contains sensitive corpora, such as medical records or legal documents.<n>We present DP-KSA, a novel privacy-preserving RAG algorithm that integrates DP using the propose-test-release paradigm.
  arXiv  Detail & Related papers  (2026-02-16T00:52:57Z)
• Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search [62.1546099504045]
  We propose a dual-phase test-time scaling framework that separates reasoning into planning and execution.<n>Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately.<n> Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing computation redundant.
  arXiv  Detail & Related papers  (2025-09-29T19:27:23Z)
• Auto-Formulating Dynamic Programming Problems with Large Language Models [4.693833469789685]
  We introduce DP-Bench, the first benchmark covering a wide range of textbook-level DP problems to enable systematic evaluation.<n>Central to DPLM's effectiveness is DualReflect, our novel synthetic data generation pipeline, designed to scale up training data from a limited set of initial examples.<n>Our results reveal a key insight: backward generation is favored in low-data regimes for its strong correctness guarantees, while forward generation becomes increasingly valuable at scale for introducing diverse formulations.
  arXiv  Detail & Related papers  (2025-07-15T21:09:43Z)
• Fractional Reasoning via Latent Steering Vectors Improves Inference Time Compute [60.151643048803145]
  We propose Fractional Reasoning, a framework that enables continuous control over reasoning intensity at inference time.<n>Our method operates by extracting the latent steering vector associated with deeper reasoning and reapplying it with a tunable scaling factor.<n> Experiments on GSM8K, MATH500, and GPQA demonstrate that Fractional Reasoning consistently improves performance across diverse reasoning tasks and models.
  arXiv  Detail & Related papers  (2025-06-18T21:15:59Z)
• PixelThink: Towards Efficient Chain-of-Pixel Reasoning [70.32510083790069]
  PixelThink is a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty.<n>It learns to compress reasoning length in accordance with scene complexity and predictive confidence.<n> Experimental results demonstrate that the proposed approach improves both reasoning efficiency and overall segmentation performance.
  arXiv  Detail & Related papers  (2025-05-29T17:55:49Z)
• Synthesizing Tight Privacy and Accuracy Bounds via Weighted Model Counting [5.552645730505715]
  Two core challenges are finding expressive, compact, and efficient encodings of distributions of DP algorithms. We address the first challenge by developing a method for tight privacy and accuracy bound synthesis. We develop a framework for leveraging inherent symmetries in DP algorithms.
  arXiv  Detail & Related papers  (2024-02-26T19:29:46Z)
• GEML: A Grammar-based Evolutionary Machine Learning Approach for Design-Pattern Detection [7.018591019975254]
  Design patterns (DPs) are recognised as a good practice in software development. The lack of appropriate documentation often hampers traceability, and their benefits are blurred among thousands of lines of code. We propose GEML, a novel detection approach based on evolutionary machine learning using software properties of diverse nature.
  arXiv  Detail & Related papers  (2024-01-13T11:05:24Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy [22.906644117887133]
  Differential Privacy (DP) has become a gold standard for making formal statements about data anonymization. The adoption of DP is hindered by limited practical guidance of what DP protection entails, what privacy guarantees to aim for, and the difficulty of achieving good privacy-utility-computation trade-offs for ML models. This work is a self-contained guide that gives an in-depth overview of the field of DP ML and presents information about achieving the best possible DP ML model with rigorous privacy guarantees.
  arXiv  Detail & Related papers  (2023-03-01T16:56:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.