Predicting potentially abusive clauses in Chilean terms of services with natural language processing

• URL: http://arxiv.org/abs/2502.00865v2
• Date: Mon, 05 May 2025 18:02:07 GMT
• Title: Predicting potentially abusive clauses in Chilean terms of services with natural language processing
• Authors: Christoffer Loeffler, Andrea Martínez Freile, Tomás Rey Pizarro,
• Abstract summary: This study addresses the growing concern of information asymmetry in consumer contracts, exacerbated by the proliferation of online services with complex Terms of Service that are rarely even read.<n>We introduce a new methodology and a substantial dataset addressing this gap.<n>We propose a novel annotation scheme with four categories and a total of 20 classes, and apply it on 50 online Terms of Service used in Chile.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This study addresses the growing concern of information asymmetry in consumer contracts, exacerbated by the proliferation of online services with complex Terms of Service that are rarely even read. Even though research on automatic analysis methods is conducted, the problem is aggravated by the general focus on English-language Machine Learning approaches and on major jurisdictions, such as the European Union. We introduce a new methodology and a substantial dataset addressing this gap. We propose a novel annotation scheme with four categories and a total of 20 classes, and apply it on 50 online Terms of Service used in Chile. Our evaluation of transformer-based models highlights how factors like language- and/or domain-specific pre-training, few-shot sample size, and model architecture affect the detection and classification of potentially abusive clauses. Results show a large variability in performance for the different tasks and models, with the highest macro-F1 scores for the detection task ranging from 79% to 89% and micro-F1 scores up to 96%, while macro-F1 scores for the classification task range from 60% to 70% and micro-F1 scores from 64% to 80%. Notably, this is the first Spanish-language multi-label classification dataset for legal clauses, applying Chilean law and offering a comprehensive evaluation of Spanish-language models in the legal domain. Our work lays the ground for future research in method development for rarely considered legal analysis and potentially leads to practical applications to support consumers in Chile and Latin America as a whole.

Related papers

• When Scale Meets Diversity: Evaluating Language Models on Fine-Grained Multilingual Claim Verification [14.187153195380668]
  Large language models have remarkable capabilities across many NLP tasks, but their effectiveness for multilingual claim verification with nuanced classification schemes remains understudied.<n>We evaluate five state-of-the-art language models on the X-Fact dataset, which spans 25 languages with seven distinct veracity categories.<n>Surprisingly, we find that XLM-R substantially outperforms all tested LLMs, achieving 57.7% macro-F1 compared to the best LLM performance of 16.9%.
  arXiv  Detail & Related papers  (2025-07-28T10:49:04Z)
• Evaluation of End-to-End Continuous Spanish Lipreading in Different Data Conditions [0.0]
  This paper presents noticeable advances in automatic continuous lipreading for Spanish.<n> Experiments are conducted on two corpora of disparate nature, reaching state-of-the-art results.<n>A rigorous error analysis is carried out to investigate the different factors that could affect the learning of the automatic system.
  arXiv  Detail & Related papers  (2025-02-01T15:48:20Z)
• CLAIR-A: Leveraging Large Language Models to Judge Audio Captions [73.51087998971418]
  evaluating machine-generated audio captions is a complex task that requires considering diverse factors. We propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models. In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics.
  arXiv  Detail & Related papers  (2024-09-19T17:59:52Z)
• Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination [35.88131356701857]
  This dataset consists of 1003 multiple-choice questions of university entrance level exams in Spanish and English.<n>A selection of current open-source and proprietary models are evaluated in a uniform zero-shot experimental setting.
  arXiv  Detail & Related papers  (2024-09-19T13:13:07Z)
• The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights [108.40766216456413]
  We propose a question alignment framework to bridge the gap between large language models' English and non-English performance. Experiment results show it can boost multilingual performance across diverse reasoning scenarios, model families, and sizes. We analyze representation space, generated response and data scales, and reveal how question translation training strengthens language alignment within LLMs.
  arXiv  Detail & Related papers  (2024-05-02T14:49:50Z)
• Resolving Legalese: A Multilingual Exploration of Negation Scope Resolution in Legal Documents [3.8467652838774873]
  complexity of legal texts and lack of annotated in-domain negation corpora pose challenges for state-of-the-art (SotA) models. Our experiments demonstrate that models pre-trained without legal data underperform in the task of negation scope resolution. We release a new set of annotated court decisions in German, French, and Italian and use it to improve negation scope resolution in both zero-shot and multilingual settings.
  arXiv  Detail & Related papers  (2023-09-15T18:38:06Z)
• A User-Centered Evaluation of Spanish Text Simplification [6.046875672600245]
  We present an evaluation of text simplification (TS) in Spanish for a production system. We compare the most prevalent Spanish-specific readability scores with neural networks, and show that the latter are consistently better at predicting user preferences regarding TS. We release the corpora in our evaluation to the broader community with the hopes of pushing forward the state-of-the-art in Spanish natural language processing.
  arXiv  Detail & Related papers  (2023-08-15T03:49:59Z)
• Automated Refugee Case Analysis: An NLP Pipeline for Supporting Legal Practitioners [0.0]
  We introduce an end-to-end pipeline for retrieving, processing, and extracting targeted information from legal cases. We investigate an under-studied legal domain with a case study on refugee law in Canada.
  arXiv  Detail & Related papers  (2023-05-24T19:37:23Z)
• Automated stance detection in complex topics and small languages: the challenging case of immigration in polarizing news media [0.0]
  This paper explores the applicability of large language models for automated stance detection in a challenging scenario. It involves a morphologically complex, lower-resource language, and a socio-culturally complex topic, immigration. If the approach works in this case, it can be expected to perform as well or better in less demanding scenarios.
  arXiv  Detail & Related papers  (2023-05-22T13:56:35Z)
• Holistic Evaluation of Language Models [183.94891340168175]
  Language models (LMs) are becoming the foundation for almost all major language technologies, but their capabilities, limitations, and risks are not well understood. We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
  arXiv  Detail & Related papers  (2022-11-16T18:51:34Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• On Cross-Lingual Retrieval with Multilingual Text Encoders [51.60862829942932]
  We study the suitability of state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. We benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR experiments. We evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we learn to rank) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments.
  arXiv  Detail & Related papers  (2021-12-21T08:10:27Z)
• AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages [75.08199398141744]
  We present AmericasNLI, an extension of XNLI (Conneau et al.), to 10 indigenous languages of the Americas. We conduct experiments with XLM-R, testing multiple zero-shot and translation-based approaches. We find that XLM-R's zero-shot performance is poor for all 10 languages, with an average performance of 38.62%.
  arXiv  Detail & Related papers  (2021-04-18T05:32:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.