AI Research is not Magic, it has to be Reproducible and Responsible: Challenges in the AI field from the Perspective of its PhD Students

• URL: http://arxiv.org/abs/2408.06847v1
• Date: Tue, 13 Aug 2024 12:19:02 GMT
• Title: AI Research is not Magic, it has to be Reproducible and Responsible: Challenges in the AI field from the Perspective of its PhD Students
• Authors: Andrea Hrckova, Jennifer Renoux, Rafael Tolosana Calasanz, Daniela Chuda, Martin Tamajka, Jakub Simko,
• Abstract summary: We surveyed 28 AI doctoral candidates from 13 European countries. Challenges underscore the findability and quality of AI resources such as datasets, models, and experiments. There is need for immediate adoption of responsible and reproducible AI research practices.
• Score: 1.1922075410173798
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: With the goal of uncovering the challenges faced by European AI students during their research endeavors, we surveyed 28 AI doctoral candidates from 13 European countries. The outcomes underscore challenges in three key areas: (1) the findability and quality of AI resources such as datasets, models, and experiments; (2) the difficulties in replicating the experiments in AI papers; (3) and the lack of trustworthiness and interdisciplinarity. From our findings, it appears that although early stage AI researchers generally tend to share their AI resources, they lack motivation or knowledge to engage more in dataset and code preparation and curation, and ethical assessments, and are not used to cooperate with well-versed experts in application domains. Furthermore, we examine existing practices in data governance and reproducibility both in computer science and in artificial intelligence. For instance, only a minority of venues actively promote reproducibility initiatives such as reproducibility evaluations. Critically, there is need for immediate adoption of responsible and reproducible AI research practices, crucial for society at large, and essential for the AI research community in particular. This paper proposes a combination of social and technical recommendations to overcome the identified challenges. Socially, we propose the general adoption of reproducibility initiatives in AI conferences and journals, as well as improved interdisciplinary collaboration, especially in data governance practices. On the technical front, we call for enhanced tools to better support versioning control of datasets and code, and a computing infrastructure that facilitates the sharing and discovery of AI resources, as well as the sharing, execution, and verification of experiments.

Related papers

• OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI [73.75520820608232]
  We introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities. These challenges encompass a wide range of disciplines spanning seven fields and 62 international Olympic competitions, rigorously examined for data leakage. Our evaluations reveal that even advanced models like GPT-4o only achieve a 39.97% overall accuracy, illustrating current AI limitations in complex reasoning and multimodal integration.
  arXiv  Detail & Related papers  (2024-06-18T16:20:53Z)
• Beyond principlism: Practical strategies for ethical AI use in research practices [0.0]
  The rapid adoption of generative artificial intelligence in scientific research has outpaced the development of ethical guidelines. Existing approaches offer little practical guidance for addressing ethical challenges of AI in scientific research practices. I propose a user-centered, realism-inspired approach to bridge the gap between abstract principles and day-to-day research practices.
  arXiv  Detail & Related papers  (2024-01-27T03:53:25Z)
• AI for social science and social science of AI: A Survey [47.5235291525383]
  Recent advancements in artificial intelligence have sparked a rethinking of artificial general intelligence possibilities. The increasing human-like capabilities of AI are also attracting attention in social science research.
  arXiv  Detail & Related papers  (2024-01-22T10:57:09Z)
• Investigating Responsible AI for Scientific Research: An Empirical Study [4.597781832707524]
  The push for Responsible AI (RAI) in such institutions underscores the increasing emphasis on integrating ethical considerations within AI design and development. This paper aims to assess the awareness and preparedness regarding the ethical risks inherent in AI design and development. Our results have revealed certain knowledge gaps concerning ethical, responsible, and inclusive AI, with limitations in awareness of the available AI ethics frameworks.
  arXiv  Detail & Related papers  (2023-12-15T06:40:27Z)
• On the Opportunities of Green Computing: A Survey [80.21955522431168]
  Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades. The needs for high computing power brings higher carbon emission and undermines research fairness. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
  arXiv  Detail & Related papers  (2023-11-01T11:16:41Z)
• The ethical ambiguity of AI data enrichment: Measuring gaps in research ethics norms and practices [2.28438857884398]
  This study explores how, and to what extent, comparable research ethics requirements and norms have developed for AI research and data enrichment. Leading AI venues have begun to establish protocols for human data collection, but these are are inconsistently followed by authors.
  arXiv  Detail & Related papers  (2023-06-01T16:12:55Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Stakeholder Participation in AI: Beyond "Add Diverse Stakeholders and Stir" [76.44130385507894]
  This paper aims to ground what we dub a 'participatory turn' in AI design by synthesizing existing literature on participation and through empirical analysis of its current practices. Based on our literature synthesis and empirical research, this paper presents a conceptual framework for analyzing participatory approaches to AI design.
  arXiv  Detail & Related papers  (2021-11-01T17:57:04Z)
• Empowering Local Communities Using Artificial Intelligence [70.17085406202368]
  It has become an important topic to explore the impact of AI on society from a people-centered perspective. Previous works in citizen science have identified methods of using AI to engage the public in research. This article discusses the challenges of applying AI in Community Citizen Science.
  arXiv  Detail & Related papers  (2021-10-05T12:51:11Z)
• A narrowing of AI research? [0.0]
  We study the evolution of the thematic diversity of AI research in academia and the private sector. We measure the influence of private companies in AI research through the citations they receive and their collaborations with other institutions.
  arXiv  Detail & Related papers  (2020-09-22T08:23:56Z)
• Report prepared by the Montreal AI Ethics Institute (MAIEI) on Publication Norms for Responsible AI [0.0]
  Montreal AI Ethics Institute co-hosted two public consultations with the Partnership on AI in May 2020. The meetups examined potential publication norms for responsible AI. MAIEI provides six initial recommendations.
  arXiv  Detail & Related papers  (2020-09-15T17:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.