An Empirical Study on Compliance with Ranking Transparency in the Software Documentation of EU Online Platforms

• URL: http://arxiv.org/abs/2312.14794v2
• Date: Tue, 23 Jan 2024 13:07:38 GMT
• Title: An Empirical Study on Compliance with Ranking Transparency in the Software Documentation of EU Online Platforms
• Authors: Francesco Sovrano, Micha\"el Lognoul, Alberto Bacchelli
• Abstract summary: This study empirically evaluate the compliance of six major platforms (Amazon, Bing, Booking, Google, Tripadvisor, and Yahoo) We introduce and test automated compliance assessment tools based on ChatGPT and information retrieval technology. Our findings could help enhance regulatory compliance and align with the United Nations Sustainable Development Goal 10.3.
• Score: 7.461555266672227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compliance with the European Union's Platform-to-Business (P2B) Regulation is challenging for online platforms, and assessing their compliance can be difficult for public authorities. This is partly due to the lack of automated tools for assessing the information (e.g., software documentation) platforms provide concerning ranking transparency. Our study tackles this issue in two ways. First, we empirically evaluate the compliance of six major platforms (Amazon, Bing, Booking, Google, Tripadvisor, and Yahoo), revealing substantial differences in their documentation. Second, we introduce and test automated compliance assessment tools based on ChatGPT and information retrieval technology. These tools are evaluated against human judgments, showing promising results as reliable proxies for compliance assessments. Our findings could help enhance regulatory compliance and align with the United Nations Sustainable Development Goal 10.3, which seeks to reduce inequality, including business disparities, on these platforms.

Related papers

• Adaptive PII Mitigation Framework for Large Language Models [2.694044579874688]
  This paper introduces an adaptive system for mitigating risk of Personally Identifiable Information (PII) and Sensitive Personal Information (SPI) The system uses advanced NLP techniques, context-aware analysis, and policy-driven masking to ensure regulatory compliance. Benchmarks highlight the system's effectiveness, with an F1 score of 0.95 for Passport Numbers.
  arXiv  Detail & Related papers  (2025-01-21T19:22:45Z)
• Metamorphic Debugging for Accountable Software [8.001739956625483]
  Translating legalese into formal specifications is one challenge. Lack of a definitive 'truth' for queries (the oracle problem) is another. We propose that these challenges can be tackled by focusing on relational specifications.
  arXiv  Detail & Related papers  (2024-09-24T14:45:13Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• Auditing Recommender Systems -- Putting the DSA into practice with a risk-scenario-based approach [5.875955066693127]
  European Union's Digital Services Act requires platforms to make algorithmic systems more transparent and follow due diligence obligations. These requirements constitute an important legislative step towards mitigating the systemic risks posed by online platforms. But the DSA lacks concrete guidelines to operationalise a viable audit process. This void could foster the spread of 'audit-washing', that is, platforms exploiting audits to legitimise their practices and neglect responsibility.
  arXiv  Detail & Related papers  (2023-02-09T10:48:37Z)
• Having your Privacy Cake and Eating it Too: Platform-supported Auditing of Social Media Algorithms for Public Interest [70.02478301291264]
  Social media platforms curate access to information and opportunities, and so play a critical role in shaping public discourse. Prior studies have used black-box methods to show that these algorithms can lead to biased or discriminatory outcomes. We propose a new method for platform-supported auditing that can meet the goals of the proposed legislation.
  arXiv  Detail & Related papers  (2022-07-18T17:32:35Z)
• Adversarial Scrutiny of Evidentiary Statistical Software [32.962815960406196]
  U.S. criminal legal system increasingly relies on software output to convict and incarcerate people. We propose robust adversarial testing as an audit framework to examine the validity of evidentiary statistical software.
  arXiv  Detail & Related papers  (2022-06-19T02:08:42Z)
• Integrating Rankings into Quantized Scores in Peer Review [61.27794774537103]
  In peer review, reviewers are usually asked to provide scores for the papers. To mitigate this issue, conferences have started to ask reviewers to additionally provide a ranking of the papers they have reviewed. There are no standard procedure for using this ranking information and Area Chairs may use it in different ways. We take a principled approach to integrate the ranking information into the scores.
  arXiv  Detail & Related papers  (2022-04-05T19:39:13Z)
• Fairness Score and Process Standardization: Framework for Fairness Certification in Artificial Intelligence Systems [0.4297070083645048]
  We propose a novel Fairness Score to measure the fairness of a data-driven AI system. It will also provide a framework to operationalise the concept of fairness and facilitate the commercial deployment of such systems.
  arXiv  Detail & Related papers  (2022-01-10T15:45:12Z)
• Benchmarks for Deep Off-Policy Evaluation [152.28569758144022]
  We present a collection of policies that can be used for benchmarking off-policy evaluation. The goal of our benchmark is to provide a standardized measure of progress that is motivated from a set of principles. We provide open-source access to our data and code to foster future research in this area.
  arXiv  Detail & Related papers  (2021-03-30T18:09:33Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)
• Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics [64.88815792555451]
  We show that current methods for judging metrics are highly sensitive to the translations used for assessment. We develop a method for thresholding performance improvement under an automatic metric against human judgements.
  arXiv  Detail & Related papers  (2020-06-11T09:12:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.