PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Data

• URL: http://arxiv.org/abs/2609.10612v1
• Date: Tue, 08 Sep 2026 14:12:48 GMT
• Title: PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Data
• Abstract summary: Differentially private (DP) synthetic data can support system development and educational research while reducing exposure of individual learners.<n>We introduce PEARL (Privacy-Equivalence Audit and Release Ledger), which approves a DP synthetic educational dataset only when it passes all required checks of validity, privacy protection, predictive usefulness, and suitability for the intended educational task.
• Score: 3.8111836974655957
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized learning systems rely on real learner data, including performance, behavior, and demographic information, but these data are highly privacy-sensitive. Differentially private (DP) synthetic data can support system development and educational research while reducing exposure of individual learners. Existing evaluations, however, assess privacy and predictive usefulness separately, without determining whether synthetic learner data remain usable for the intended personalized learning task. We introduce PEARL (Privacy-Equivalence Audit and Release Ledger), which approves a DP synthetic educational dataset only when it passes all required checks of validity, privacy protection, predictive usefulness, and suitability for the intended educational task, while recording why each rejected dataset fails. Across 96 study settings, each defined by a dataset, data-generation method, privacy budget, and random seed, only 12 produced synthetic datasets that passed all applicable PEARL checks. Many privacy-protected datasets were rejected for omitting important outcome groups, such as withdrawn students, or for failing to preserve the order of learning activities. Fairness analysis further showed that some datasets passing privacy and predictive-usefulness checks still yielded unequal at-risk prediction performance across groups defined by disability and socioeconomic background. Moreover, Deep Knowledge Tracing and Self-Attentive Knowledge Tracing learned no meaningful next-response patterns from any tested synthetic knowledge-tracing dataset, showing that privacy protection alone does not guarantee usefulness for dropout prediction, knowledge tracing, or adaptive tutoring.

Related papers

• Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data [56.86147283213652]
  We present a customizable empirical auditing framework designed to detect and explain data disclosures.<n>Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures"-where the system incidentally generates a user's data.
  arXiv  Detail & Related papers  (2026-06-15T16:54:02Z)
• PrivATE: Differentially Private Average Treatment Effect Estimation for Observational Data [49.35645194884526]
  We introduce PrivATE, a practical ATE estimation framework that ensures differential privacy.<n>We design two levels (i.e., label-level and sample-level) of privacy protection in PrivATE to accommodate different privacy requirements.<n>PrivATE effectively balances noise-induced error and matching error, leading to a more accurate estimate of ATE.
  arXiv  Detail & Related papers  (2025-12-16T16:30:07Z)
• Aim High, Stay Private: Differentially Private Synthetic Data Enables Public Release of Behavioral Health Information with High Utility [2.1715431485081593]
  Differential Privacy (DP) provides formal guarantees against re-identification risks.<n>We generate DP synthetic data for Phase 1 of the Lived Experiences Measured Using Rings Study (LEMURS)<n>We evaluate the utility of the synthetic data using a framework informed by actual uses of the LEMURS dataset.
  arXiv  Detail & Related papers  (2025-06-30T15:58:34Z)
• Differential Privacy in Machine Learning: From Symbolic AI to LLMs [49.1574468325115]
  Differential privacy provides a formal framework to mitigate privacy risks.<n>It ensures that the inclusion or exclusion of any single data point does not significantly alter the output of an algorithm.
  arXiv  Detail & Related papers  (2025-06-13T11:30:35Z)
• PASS: Private Attributes Protection with Stochastic Data Substitution [46.38957234350463]
  Various studies have been proposed to protect private attributes by removing them from the data while maintaining the utilities of the data for downstream tasks.<n> PASS is designed to substitute the original sample with another one according to certain probabilities, which is trained with a novel loss function.<n>The comprehensive evaluation of PASS on various datasets of different modalities, including facial images, human activity sensory signals, and voice recording datasets, substantiates PASS's effectiveness and generalizability.
  arXiv  Detail & Related papers  (2025-06-08T22:48:07Z)
• Tabular Data Synthesis with Differential Privacy: A Survey [24.500349285858597]
  Data sharing is a prerequisite for collaborative innovation, enabling organizations to leverage diverse datasets for deeper insights. Data synthesis tackles this by generating artificial datasets that preserve the statistical characteristics of real data. Differentially private data synthesis has emerged as a promising approach to privacy-aware data sharing.
  arXiv  Detail & Related papers  (2024-11-04T06:32:48Z)
• Approximate, Adapt, Anonymize (3A): a Framework for Privacy Preserving Training Data Release for Machine Learning [3.29354893777827]
  We introduce a data release framework, 3A (Approximate, Adapt, Anonymize), to maximize data utility for machine learning. We present experimental evidence showing minimal discrepancy between performance metrics of models trained on real versus privatized datasets.
  arXiv  Detail & Related papers  (2023-07-04T18:37:11Z)
• "You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learning [78.70083858195906]
  We propose differentially private mechanisms to measure differences in performance across groups while protecting the privacy of group membership. Our results show that, contrary to what prior work suggested, protecting privacy is not necessarily in conflict with identifying performance disparities of federated models.
  arXiv  Detail & Related papers  (2022-06-24T09:46:43Z)
• SF-PATE: Scalable, Fair, and Private Aggregation of Teacher Ensembles [50.90773979394264]
  This paper studies a model that protects the privacy of individuals' sensitive information while also allowing it to learn non-discriminatory predictors. A key characteristic of the proposed model is to enable the adoption of off-the-selves and non-private fair models to create a privacy-preserving and fair model.
  arXiv  Detail & Related papers  (2022-04-11T14:42:54Z)
• Towards a Data Privacy-Predictive Performance Trade-off [2.580765958706854]
  We evaluate the existence of a trade-off between data privacy and predictive performance in classification tasks. Unlike previous literature, we confirm that the higher the level of privacy, the higher the impact on predictive performance.
  arXiv  Detail & Related papers  (2022-01-13T21:48:51Z)
• Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach [54.32266555843765]
  This paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints.
  arXiv  Detail & Related papers  (2020-09-26T10:50:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.