Autonomation, not Automation: Activities and Needs of Fact-checkers as a Basis for Designing Human-Centered AI Systems

• URL: http://arxiv.org/abs/2211.12143v2
• Date: Tue, 13 Aug 2024 11:15:09 GMT
• Title: Autonomation, not Automation: Activities and Needs of Fact-checkers as a Basis for Designing Human-Centered AI Systems
• Authors: Andrea Hrckova, Robert Moro, Ivan Srba, Jakub Simko, Maria Bielikova,
• Abstract summary: We conducted in-depth interviews with Central European fact-checkers. Our contributions include an in-depth examination of the variability of fact-checking work in non-English speaking regions. Thanks to the interdisciplinary collaboration, we extend the fact-checking process in AI research by three additional stages.
• Score: 1.7925621668797338
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To mitigate the negative effects of false information more effectively, the development of Artificial Intelligence (AI) systems assisting fact-checkers is needed. Nevertheless, the lack of focus on the needs of these stakeholders results in their limited acceptance and skepticism toward automating the whole fact-checking process. In this study, we conducted semi-structured in-depth interviews with Central European fact-checkers. Their activities and problems were analyzed using iterative content analysis. The most significant problems were validated with a survey of European fact-checkers, in which we collected 24 responses from 20 countries, i.e., 62\% of active European signatories of the International Fact-Checking Network (IFCN). Our contributions include an in-depth examination of the variability of fact-checking work in non-English speaking regions, which still remained largely uncovered. By aligning them with the knowledge from prior studies, we created conceptual models that help understand the fact-checking processes. Thanks to the interdisciplinary collaboration, we extend the fact-checking process in AI research by three additional stages. In addition, we mapped our findings on the fact-checkers' activities and needs to the relevant tasks for AI research. The new opportunities identified for AI researchers and developers have implications for the focus of AI research in this domain.

Related papers

• ML Research Benchmark [0.0]
  We present the ML Research Benchmark (MLRB), comprising 7 competition-level tasks derived from recent machine learning conference tracks. This paper introduces a novel benchmark and evaluates it using agent scaffolds powered by frontier models, including Claude-3 and GPT-4o. The results indicate that the Claude-3.5 Sonnet agent performs best across our benchmark, excelling in planning and developing machine learning models.
  arXiv  Detail & Related papers  (2024-10-29T21:38:42Z)
• AI Research is not Magic, it has to be Reproducible and Responsible: Challenges in the AI field from the Perspective of its PhD Students [1.1922075410173798]
  We surveyed 28 AI doctoral candidates from 13 European countries. Challenges underscore the findability and quality of AI resources such as datasets, models, and experiments. There is need for immediate adoption of responsible and reproducible AI research practices.
  arXiv  Detail & Related papers  (2024-08-13T12:19:02Z)
• OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI [73.75520820608232]
  We introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities. These challenges encompass a wide range of disciplines spanning seven fields and 62 international Olympic competitions, rigorously examined for data leakage. Our evaluations reveal that even advanced models like GPT-4o only achieve a 39.97% overall accuracy, illustrating current AI limitations in complex reasoning and multimodal integration.
  arXiv  Detail & Related papers  (2024-06-18T16:20:53Z)
• The Impact and Opportunities of Generative AI in Fact-Checking [12.845170214324662]
  Generative AI appears poised to transform white collar professions, with more than 90% of Fortune 500 companies using OpenAI's flagship GPT models. But how will such technologies impact organizations whose job is to verify and report factual information? We conducted 30 interviews with N=38 participants working at 29 fact-checking organizations across six continents.
  arXiv  Detail & Related papers  (2024-05-24T23:58:01Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• The Foundations of Computational Management: A Systematic Approach to Task Automation for the Integration of Artificial Intelligence into Existing Workflows [55.2480439325792]
  This article introduces Computational Management, a systematic approach to task automation. The article offers three easy step-by-step procedures to begin the process of implementing AI within a workflow.
  arXiv  Detail & Related papers  (2024-02-07T01:45:14Z)
• Assessing AI Impact Assessments: A Classroom Study [14.768235460961876]
  Artificial Intelligence Impact Assessments ("AIIAs"), a family of tools that provide structured processes to imagine the possible impacts of a proposed AI system, have become an increasingly popular proposal to govern AI systems. Recent efforts from government or private-sector organizations have proposed many diverse instantiations of AIIAs, which take a variety of forms ranging from open-ended questionnaires to graded score-cards. We conduct a classroom study at a large research-intensive university (R1) in an elective course focused on the societal and ethical implications of AI. We find preliminary evidence that impact assessments can influence participants' perceptions of the potential
  arXiv  Detail & Related papers  (2023-11-19T01:00:59Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• Towards AI-Empowered Crowdsourcing [27.0404686687184]
  We propose a taxonomy which divides AI-Empowered Crowdsourcing into three major areas: task delegation, motivating workers, and quality control. We discuss the limitations and insights, and curate the challenges of doing research in each of these areas to highlight promising future research directions.
  arXiv  Detail & Related papers  (2022-12-28T05:06:55Z)
• Characterising Research Areas in the field of AI [68.8204255655161]
  We identified the main conceptual themes by performing clustering analysis on the co-occurrence network of topics. The results highlight the growing academic interest in research themes like deep learning, machine learning, and internet of things.
  arXiv  Detail & Related papers  (2022-05-26T16:30:30Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.