Enabling Multilingual Privacy Policy Audits: Large-Scale Analysis of Spanish Mobile Apps

• URL: http://arxiv.org/abs/2607.18424v1
• Date: Mon, 20 Jul 2026 18:13:49 GMT
• Title: Enabling Multilingual Privacy Policy Audits: Large-Scale Analysis of Spanish Mobile Apps
• Abstract summary: This paper examines whether large language models (LLMs) can extend privacy policy analysis beyond English without requiring language-specific adaptation.<n>We assemble an evaluation corpus spanning all 24 official EU languages from translated versions of two established expert-annotated datasets.<n>Our results indicate how English-only privacy audits can systematically obfuscate transparency gaps in multilingual environments.
• Score: 0.5707369719866678
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated analyses of privacy policies enable large-scale assessments of transparency in digital ecosystems, yet existing auditing pipelines remain predominantly English-centric. This limits their ability to systematically evaluate multilingual environments, as in the European Union, where many services disclose privacy practices only in local languages. This paper examines whether large language models (LLMs) can extend privacy policy analysis beyond English without requiring language-specific adaptation, thus empowering large-scale auditing in linguistically diverse app ecosystems. We assemble an evaluation corpus spanning all 24 official EU languages from translated versions of two established expert-annotated datasets (OPP-115 and MAPP) and assess translation fidelity through automated metrics and targeted legal-expert review. Our LLM-based classifier for identifying categories of personal data collection achieves stable cross-lingual performance, with macro-F1 scores ranging between 0.91 and 0.94. We then leverage this capability in a large-scale audit of 2,611 Android applications from the Spanish Google Play Store. Combining multilingual privacy policy analysis with the evaluation of corresponding privacy labels and runtime network traffic exposes an important linguistic barrier: public-sector apps predominantly provide privacy policies in Spanish, whereas popular commercial apps mostly provide them in English. We reveal systematic discrepancies between declared and observed practices, especially in public-sector apps. Overall, our results indicate how English-only privacy audits can systematically obfuscate transparency gaps in multilingual environments.

Related papers

• Multilingual Refusal Alignment for Safer Large Language Models [53.64286756804503]
  We investigate whether single-language alignment transfers cross-lingually, how language consistency is preserved during training, and the resulting trade-offs with general knowledge capabilities.<n>We introduce RefusEU, a novel refusal alignment dataset covering 12 European languages, including a dedicated test set for evaluating current state-of-the-art models.<n>Our controlled Direct Preference Optimization (DPO) experiments provide two key insights: aligning models exclusively in English is insufficient to ensure cross-lingual safety, even for the same harm categories, whereas training on multilingual datasets can improve safety without degrading general performance, as measured by the Global MMLU benchmark.
  arXiv  Detail & Related papers  (2026-04-24T09:29:14Z)
• An Empirical Recipe for Universal Phone Recognition [63.45609714127985]
  Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks.<n>We present PhoneticXEUS -- trained on large-scale multilingual data.<n>We achieve state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER)
  arXiv  Detail & Related papers  (2026-03-30T22:12:48Z)
• LangGPS: Language Separability Guided Data Pre-Selection for Joint Multilingual Instruction Tuning [49.22807995935406]
  Joint multilingual instruction tuning is a widely adopted approach to improve the multilingual instruction-following ability and downstream performance of large language models (LLMs)<n>Existing selection methods, often based on features like text quality, diversity, or task relevance, typically overlook the intrinsic linguistic structure of multilingual data.<n>We propose LangGPS, a lightweight two-stage pre-selection framework guided by language separability which quantifies how well samples in different languages can be distinguished in the model's representation space.
  arXiv  Detail & Related papers  (2025-11-13T12:02:32Z)
• The Model's Language Matters: A Comparative Privacy Analysis of LLMs [0.3441021278275805]
  We quantify six linguistic indicators and evaluate three attack vectors: extraction, counterfactual memorization, and membership inference.<n>Italian exhibits the strongest leakage, while English shows higher membership separability.<n>French and Spanish display greater resilience due to higher morphological complexity.
  arXiv  Detail & Related papers  (2025-10-09T20:59:42Z)
• Multilingual Self-Taught Faithfulness Evaluators [11.200203292660758]
  Self-Taught Evaluators for Multilingual Faithfulness is a framework that learns exclusively from synthetic multilingual summarization data.<n>Our framework shows improvements over existing baselines, including state-of-the-art English evaluators and machine translation-based approaches.
  arXiv  Detail & Related papers  (2025-07-28T12:01:59Z)
• The AI Language Proficiency Monitor -- Tracking the Progress of LLMs on Multilingual Benchmarks [0.0]
  We introduce the AI Language Monitor, a comprehensive benchmark that assesses large language models (LLMs) performance across up to 200 languages.<n>Our benchmark aggregates diverse tasks including translation, question answering, math, and reasoning, using datasets such as FLORES+, MMLU, GSM8K, TruthfulQA, and ARC.<n>We provide an open-source, auto-updating leaderboard and dashboard that supports researchers, developers, and policymakers in identifying strengths and gaps in model performance.
  arXiv  Detail & Related papers  (2025-07-11T12:38:02Z)
• MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety [56.77103365251923]
  Large Language Models (LLMs) are susceptible to adversarial attacks such as jailbreaking.<n>This vulnerability is exacerbated in multilingual settings, where multilingual safety-aligned data is often limited.<n>We introduce a multilingual guardrail with reasoning for prompt classification.
  arXiv  Detail & Related papers  (2025-04-21T17:15:06Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora.<n>But can these models relate corresponding concepts across languages, i.e., be crosslingual?<n>This study evaluates state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• Building cross-language corpora for human understanding of privacy policies [7.1707060082291925]
  This work provides a methodology for building comparable cross-language in a national language and a reference study language. We provide an application example of our methodology comparing English and Italian extending the corpus of one of the first studies about users understanding of technical terms in privacy policies.
  arXiv  Detail & Related papers  (2023-02-10T16:16:55Z)
• PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English [77.79102359580702]
  We introduce the Privacy Policy Language Understanding Evaluation benchmark, a multi-task benchmark for evaluating the privacy policy language understanding. We also collect a large corpus of privacy policies to enable privacy policy domain-specific language model pre-training. We demonstrate that domain-specific continual pre-training offers performance improvements across all tasks.
  arXiv  Detail & Related papers  (2022-12-20T05:58:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.