Domain-Specific Evaluation Strategies for AI in Journalism

• URL: http://arxiv.org/abs/2403.17911v1
• Date: Tue, 26 Mar 2024 17:47:25 GMT
• Title: Domain-Specific Evaluation Strategies for AI in Journalism
• Authors: Sachita Nishal, Charlotte Li, Nicholas Diakopoulos,
• Abstract summary: We consider different aspects of AI evaluation that can benefit from domain-specific tailoring. In doing so, we lay out a potential framework to guide AI evaluation in journalism.
• Score: 3.2566808526538873
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: News organizations today rely on AI tools to increase efficiency and productivity across various tasks in news production and distribution. These tools are oriented towards stakeholders such as reporters, editors, and readers. However, practitioners also express reservations around adopting AI technologies into the newsroom, due to the technical and ethical challenges involved in evaluating AI technology and its return on investments. This is to some extent a result of the lack of domain-specific strategies to evaluate AI models and applications. In this paper, we consider different aspects of AI evaluation (model outputs, interaction, and ethics) that can benefit from domain-specific tailoring, and suggest examples of how journalistic considerations can lead to specialized metrics or strategies. In doing so, we lay out a potential framework to guide AI evaluation in journalism, such as seen in other disciplines (e.g. law, healthcare). We also consider directions for future work, as well as how our approach might generalize to other domains.

Related papers

• AI Thinking: A framework for rethinking artificial intelligence in practice [2.9805831933488127]
  A growing range of disciplines are now involved in studying, developing, and assessing the use of AI in practice. New, interdisciplinary approaches are needed to bridge competing conceptualisations of AI in practice. I propose a novel conceptual framework called AI Thinking, which models key decisions and considerations involved in AI use across disciplinary perspectives.
  arXiv  Detail & Related papers  (2024-08-26T04:41:21Z)
• Guiding the Way: A Comprehensive Examination of AI Guidelines in Global Media [0.0]
  This study analyzes 37 AI guidelines for media purposes in 17 countries. Our analysis reveals key thematic areas, such as transparency, accountability, fairness, privacy, and the preservation of journalistic values. Results highlight shared principles and best practices that emerge from these guidelines.
  arXiv  Detail & Related papers  (2024-05-07T22:47:56Z)
• Now, Later, and Lasting: Ten Priorities for AI Research, Policy, and Practice [63.20307830884542]
  Next several decades may well be a turning point for humanity, comparable to the industrial revolution. Launched a decade ago, the project is committed to a perpetual series of studies by multidisciplinary experts. We offer ten recommendations for action that collectively address both the short- and long-term potential impacts of AI technologies.
  arXiv  Detail & Related papers  (2024-04-06T22:18:31Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• AI Fairness: from Principles to Practice [0.0]
  This paper summarizes and evaluates various approaches, methods, and techniques for pursuing fairness in AI systems. It proposes practical guidelines for defining, measuring, and preventing bias in AI.
  arXiv  Detail & Related papers  (2022-07-20T11:37:46Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Actionable Approaches to Promote Ethical AI in Libraries [7.1492901819376415]
  The widespread use of artificial intelligence (AI) in many domains has revealed numerous ethical issues. No practical guidance currently exists for libraries to plan for, evaluate, or audit the ethics of intended or deployed AI. We report on several promising approaches for promoting ethical AI that can be adapted from other contexts.
  arXiv  Detail & Related papers  (2021-09-20T16:38:49Z)
• Measuring Ethics in AI with AI: A Methodology and Dataset Construction [1.6861004263551447]
  We propose to use such newfound capabilities of AI technologies to augment our AI measuring capabilities. We do so by training a model to classify publications related to ethical issues and concerns. We highlight the implications of AI metrics, in particular their contribution towards developing trustful and fair AI-based tools and technologies.
  arXiv  Detail & Related papers  (2021-07-26T00:26:12Z)
• Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
  In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society. A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests. This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
  arXiv  Detail & Related papers  (2021-06-25T22:31:55Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.