When Machine Learning Gets Personal: Understanding Fairness of Personalized Models

• URL: http://arxiv.org/abs/2502.02786v1
• Date: Wed, 05 Feb 2025 00:17:33 GMT
• Title: When Machine Learning Gets Personal: Understanding Fairness of Personalized Models
• Authors: Louisa Cornelis, Guillermo Bernárdez, Haewon Jeong, Nina Miolane,
• Abstract summary: Personalization in machine learning involves tailoring models to individual users by incorporating personal attributes such as demographic or medical data. While personalization can improve prediction accuracy, it may also amplify biases and reduce explainability. This work introduces a unified framework to evaluate the impact of personalization on both prediction accuracy and explanation quality.
• Score: 5.002195711989324
• License:
• Abstract: Personalization in machine learning involves tailoring models to individual users by incorporating personal attributes such as demographic or medical data. While personalization can improve prediction accuracy, it may also amplify biases and reduce explainability. This work introduces a unified framework to evaluate the impact of personalization on both prediction accuracy and explanation quality across classification and regression tasks. We derive novel upper bounds for the number of personal attributes that can be used to reliably validate benefits of personalization. Our analysis uncovers key trade-offs. We show that regression models can potentially utilize more personal attributes than classification models. We also demonstrate that improvements in prediction accuracy due to personalization do not necessarily translate to enhanced explainability -- underpinning the importance to evaluate both metrics when personalizing machine learning models in critical settings such as healthcare. Validated with a real-world dataset, this framework offers practical guidance for balancing accuracy, fairness, and interpretability in personalized models.

Related papers

• Developing a Dataset-Adaptive, Normalized Metric for Machine Learning Model Assessment: Integrating Size, Complexity, and Class Imbalance [0.0]
  Traditional metrics like accuracy, F1-score, and precision are frequently used to evaluate machine learning models. A dataset-adaptive, normalized metric that incorporates dataset characteristics like size, feature dimensionality, class imbalance, and signal-to-noise ratio is presented.
  arXiv  Detail & Related papers  (2024-12-10T07:10:00Z)
• What Do Learning Dynamics Reveal About Generalization in LLM Reasoning? [83.83230167222852]
  We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy. By connecting a model's learning behavior to its generalization, pre-memorization train accuracy can guide targeted improvements to training strategies.
  arXiv  Detail & Related papers  (2024-11-12T09:52:40Z)
• Robust personnel rostering: how accurate should absenteeism predictions be? [2.265037251840661]
  Disruptions to personnel rosters caused by absenteeism often necessitate last-minute adjustments to the employees' working hours. A common strategy to mitigate the impact of such changes is to assign employees to reserve shifts. We assume a predict-then-optimize approach that uses absence predictions from a machine learning model to schedule an adequate number of reserve shifts.
  arXiv  Detail & Related papers  (2024-06-26T07:16:18Z)
• Personality Profiling: How informative are social media profiles in predicting personal information? [0.04096453902709291]
  We explore the extent to which peoples' online digital footprints can be used to profile their Myers-Briggs personality type. We compare four models: logistic regression, naive Bayes, support vector machines (SVMs) and random forests. A SVM model achieves the best accuracy of 20.95% for predicting a complete personality type.
  arXiv  Detail & Related papers  (2023-09-15T03:09:43Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Personalized Privacy Auditing and Optimization at Test Time [44.15285550981899]
  This paper asks whether it is necessary to require emphall input features for a model to return accurate predictions at test time. Under a personalized setting, each individual may need to release only a small subset of these features without impacting the final decisions. Evaluation over several learning tasks shows that individuals may be able to report as little as 10% of their information to ensure the same level of accuracy.
  arXiv  Detail & Related papers  (2023-01-31T20:16:59Z)
• Self-Distillation for Further Pre-training of Transformers [83.84227016847096]
  We propose self-distillation as a regularization for a further pre-training stage. We empirically validate the efficacy of self-distillation on a variety of benchmark datasets for image and text classification tasks.
  arXiv  Detail & Related papers  (2022-09-30T02:25:12Z)
• SF-PATE: Scalable, Fair, and Private Aggregation of Teacher Ensembles [50.90773979394264]
  This paper studies a model that protects the privacy of individuals' sensitive information while also allowing it to learn non-discriminatory predictors. A key characteristic of the proposed model is to enable the adoption of off-the-selves and non-private fair models to create a privacy-preserving and fair model.
  arXiv  Detail & Related papers  (2022-04-11T14:42:54Z)
• Model Comparison and Calibration Assessment: User Guide for Consistent Scoring Functions in Machine Learning and Actuarial Practice [0.0]
  This user guide revisits and clarifies statistical techniques to assess the calibration or adequacy of a model. It focuses mainly on the pedagogical presentation of existing results and of best practice. Results are accompanied and illustrated by two real data case studies on workers' compensation and customer churn.
  arXiv  Detail & Related papers  (2022-02-25T15:52:19Z)
• DeepCoDA: personalized interpretability for compositional health data [58.841559626549376]
  Interpretability allows the domain-expert to evaluate the model's relevance and reliability. In the healthcare setting, interpretable models should implicate relevant biological mechanisms independent of technical factors. We define personalized interpretability as a measure of sample-specific feature attribution.
  arXiv  Detail & Related papers  (2020-06-02T05:14:22Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.