FAME: Fictional Actors for Multilingual Erasure

• URL: http://arxiv.org/abs/2512.15235v1
• Date: Wed, 17 Dec 2025 09:35:05 GMT
• Title: FAME: Fictional Actors for Multilingual Erasure
• Abstract summary: We introduce FAME (Fictional Actors for Multilingual Erasure), a benchmark for evaluating Machine Unlearning across five languages.<n>FAME contains 1,000 fictional actor biographies and 20,000 question-answer pairs.<n>It enables entity-level unlearning (i.e., forgetting entire identities) and instance-level unlearning (i.e., forgetting specific facts while retaining others) across languages.
• Score: 14.63217609266396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs trained on web-scale data raise concerns about privacy and the right to be forgotten. To address these issues, Machine Unlearning provides techniques to remove specific information from trained models without retraining from scratch. However, existing benchmarks for evaluating unlearning in LLMs face two major limitations: they focus only on English and support only entity-level forgetting (removing all information about a person). We introduce FAME (Fictional Actors for Multilingual Erasure), a synthetic benchmark for evaluating Machine Unlearning across five languages: English, French, German, Italian, and Spanish. FAME contains 1,000 fictional actor biographies and 20,000 question-answer pairs. Each biography includes information on 20 topics organized into structured categories (biography, career, achievements, personal information). This design enables both entity-level unlearning (i.e., forgetting entire identities) and instance-level unlearning (i.e., forgetting specific facts while retaining others). We provide two dataset splits to support these two different unlearning scenarios and enable systematic comparison of unlearning techniques across languages. Since FAME uses entirely fictional data, it ensures that the information was never encountered during model pretraining, allowing for a controlled evaluation of unlearning methods.

Related papers

• An Empirical Recipe for Universal Phone Recognition [63.45609714127985]
  Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks.<n>We present PhoneticXEUS -- trained on large-scale multilingual data.<n>We achieve state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER)
  arXiv  Detail & Related papers  (2026-03-30T22:12:48Z)
• Speech Unlearning [14.755831733659699]
  We introduce machine unlearning for speech tasks, a novel and underexplored research problem.<n>It aims to efficiently and effectively remove the influence of specific data from trained speech models without full retraining.<n>It has important applications in privacy preservation, removal of outdated or noisy data, and bias mitigation.
  arXiv  Detail & Related papers  (2025-06-01T06:04:16Z)
• DUSK: Do Not Unlearn Shared Knowledge [19.614306360050016]
  Machine unlearning aims to remove such 'forget' data while preserving utility and information from the'retain' set.<n>We introduce DUSK, a benchmark designed to evaluate unlearning methods under realistic data overlap.
  arXiv  Detail & Related papers  (2025-05-21T07:37:35Z)
• LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning [58.98865450345401]
  We introduce LecEval, an automated metric grounded in Mayer's Cognitive Theory of Multimedia Learning.<n>LecEval assesses effectiveness using four rubrics: Content Relevance (CR), Expressive Clarity (EC), Logical Structure (LS) and Audience Engagement (AE)<n>We curate a large-scale dataset of over 2,000 slides from more than 50 online course videos, annotated with fine-grained human ratings.
  arXiv  Detail & Related papers  (2025-05-04T12:06:47Z)
• LUME: LLM Unlearning with Multitask Evaluations [106.83812472773522]
  Unlearning aims to remove copyrighted, sensitive, or private content from large language models (LLMs) without a full retraining.<n>We develop a multi-task unlearning benchmark (LUME) which features three tasks: (1) unlearn synthetically generated creative short novels, (2) unlearn synthetic biographies with sensitive information, and (3) unlearn a collection of public biographies.
  arXiv  Detail & Related papers  (2025-02-20T23:30:45Z)
• Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models [12.2496297072989]
  Machine unlearning aims to remove undesirable content from trained language models.<n>Recent work has shown that soft token attacks (STA) can successfully extract unlearned information from large language models.<n>We show that STAs can be an inadequate tool for auditing unlearning.
  arXiv  Detail & Related papers  (2025-02-20T13:22:33Z)
• How Do Multilingual Language Models Remember Facts? [50.13632788453612]
  We show that previously identified recall mechanisms in English largely apply to multilingual contexts.<n>We localize the role of language during recall, finding that subject enrichment is language-independent.<n>In decoder-only LLMs, FVs compose these two pieces of information in two separate stages.
  arXiv  Detail & Related papers  (2024-10-18T11:39:34Z)
• A Closer Look at Machine Unlearning for Large Language Models [46.245404272612795]
  Large language models (LLMs) may memorize sensitive or copyrighted content, raising privacy and legal concerns.<n>We discuss several issues in machine unlearning for LLMs and provide our insights on possible approaches.
  arXiv  Detail & Related papers  (2024-10-10T16:56:05Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora.<n>But can these models relate corresponding concepts across languages, i.e., be crosslingual?<n>This study evaluates state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• TOFU: A Task of Fictitious Unlearning for LLMs [99.92305790945507]
  Large language models trained on massive corpora of data from the web can reproduce sensitive or private data raising both legal and ethical concerns. Unlearning, or tuning models to forget information present in their training data, provides us with a way to protect private data after training. We present TOFU, a benchmark aimed at helping deepen our understanding of unlearning.
  arXiv  Detail & Related papers  (2024-01-11T18:57:12Z)
• Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models [70.82705830137708]
  We introduce Data-driven Instruction Augmentation for Language-conditioned control (DIAL) We utilize semi-language labels leveraging the semantic understanding of CLIP to propagate knowledge onto large datasets of unlabelled demonstration data. DIAL enables imitation learning policies to acquire new capabilities and generalize to 60 novel instructions unseen in the original dataset.
  arXiv  Detail & Related papers  (2022-11-21T18:56:00Z)
• Zero-Shot Cross-Lingual Transfer with Meta Learning [45.29398184889296]
  We consider the setting of training models on multiple languages at the same time, when little or no data is available for languages other than English. We show that this challenging setup can be approached using meta-learning. We experiment using standard supervised, zero-shot cross-lingual, as well as few-shot cross-lingual settings for different natural language understanding tasks.
  arXiv  Detail & Related papers  (2020-03-05T16:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.