Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond

• URL: http://arxiv.org/abs/2503.07501v1
• Date: Mon, 10 Mar 2025 16:20:29 GMT
• Title: Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond
• Authors: Qiongxiu Li, Xiaoyu Luo, Yiyi Chen, Johannes Bjerva,
• Abstract summary: We conduct a survey of existing research on trustworthy machine learning (ML) and the role of memorization.<n>We formalize three-level long-tail granularity - class imbalance, atypicality, and noise - to reveal how current frameworks misapply these levels.
• Score: 4.5689379467873925
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The role of memorization in machine learning (ML) has garnered significant attention, particularly as modern models are empirically observed to memorize fragments of training data. Previous theoretical analyses, such as Feldman's seminal work, attribute memorization to the prevalence of long-tail distributions in training data, proving it unavoidable for samples that lie in the tail of the distribution. However, the intersection of memorization and trustworthy ML research reveals critical gaps. While prior research in memorization in trustworthy ML has solely focused on class imbalance, recent work starts to differentiate class-level rarity from atypical samples, which are valid and rare intra-class instances. However, a critical research gap remains: current frameworks conflate atypical samples with noisy and erroneous data, neglecting their divergent impacts on fairness, robustness, and privacy. In this work, we conduct a thorough survey of existing research and their findings on trustworthy ML and the role of memorization. More and beyond, we identify and highlight uncharted gaps and propose new revenues in this research direction. Since existing theoretical and empirical analyses lack the nuances to disentangle memorization's duality as both a necessity and a liability, we formalize three-level long-tail granularity - class imbalance, atypicality, and noise - to reveal how current frameworks misapply these levels, perpetuating flawed solutions. By systematizing this granularity, we draw a roadmap for future research. Trustworthy ML must reconcile the nuanced trade-offs between memorizing atypicality for fairness assurance and suppressing noise for robustness and privacy guarantee. Redefining memorization via this granularity reshapes the theoretical foundation for trustworthy ML, and further affords an empirical prerequisite for models that align performance with societal trust.

Related papers

• Fairness and Robustness in Machine Unlearning [20.758637391023345]
  We focus on fairness and robustness in machine unlearning algorithms. Experiments demonstrate the vulnerability of current state-of-the-art approximated unlearning algorithms to adversarial attacks. We demonstrate that unlearning in the intermediate and last layers is sufficient and cost-effective for time and memory complexity.
  arXiv  Detail & Related papers  (2025-04-18T10:31:44Z)
• Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectives [52.863024096759816]
  Misaligned research objectives have hindered progress in adversarial robustness research over the past decade.<n>We argue that realigned objectives are necessary for meaningful progress in adversarial alignment.
  arXiv  Detail & Related papers  (2025-02-17T15:28:40Z)
• Skewed Memorization in Large Language Models: Quantification and Decomposition [23.097554469918357]
  Memorization in Large Language Models (LLMs) poses privacy and security risks, as models may unintentionally reproduce sensitive or copyrighted data.<n>This paper examines memorization in supervised fine-tuning (SFT), exploring its relationships with training duration, dataset size, and inter-sample similarity.
  arXiv  Detail & Related papers  (2025-02-03T09:23:53Z)
• Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors [74.04775677110179]
  In-context Learning (ICL) has become the primary method for performing natural language tasks with Large Language Models (LLMs) In this work, we examine whether this is the result of the aggregation used in corresponding datasets, where trying to combine low-agreement, disparate annotations might lead to annotation artifacts that create detrimental noise in the prompt. Our results indicate that aggregation is a confounding factor in the modeling of subjective tasks, and advocate focusing on modeling individuals instead.
  arXiv  Detail & Related papers  (2024-10-17T17:16:00Z)
• Undesirable Memorization in Large Language Models: A Survey [5.659933808910005]
  memorization refers to a model's tendency to store and reproduce phrases from its training data. This paper provides a taxonomy of the literature on LLM memorization, exploring it across three dimensions: granularity, retrievability, and desirability. We conclude our survey by identifying potential research topics for the near future, including methods to balance privacy and performance.
  arXiv  Detail & Related papers  (2024-10-03T16:34:46Z)
• Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale [2.50194939587674]
  dissertation: quantifying and mitigating sources of arbitiness in ML, randomness in uncertainty estimation and optimization algorithms, in order to achieve scalability without sacrificing reliability. dissertation serves as an empirical proof by example that research on reliable measurement for machine learning is intimately bound up with research in law and policy.
  arXiv  Detail & Related papers  (2024-06-13T19:29:37Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Predictive Churn with the Set of Good Models [61.00058053669447]
  This paper explores connections between two seemingly unrelated concepts of predictive inconsistency. The first, known as predictive multiplicity, occurs when models that perform similarly produce conflicting predictions for individual samples. The second concept, predictive churn, examines the differences in individual predictions before and after model updates.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
  Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones. This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
  arXiv  Detail & Related papers  (2024-01-29T18:27:52Z)
• A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification [0.491574468325115]
  We present a large-scale empirical study for the first time enabling benchmarking confidence scoring functions. The revelation of a simple softmax response baseline as the overall best performing method underlines the drastic shortcomings of current evaluation.
  arXiv  Detail & Related papers  (2022-11-28T12:25:27Z)
• Inference for Interpretable Machine Learning: Fast, Model-Agnostic Confidence Intervals for Feature Importance [1.2891210250935146]
  We develop confidence intervals for a widely-used form of machine learning interpretation: feature importance. We do so by leveraging a form of random observation and feature subsampling called minipatch ensembles. Our approach is fast as computations needed for inference come nearly for free as part of the ensemble learning process.
  arXiv  Detail & Related papers  (2022-06-05T03:14:48Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.