`It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges
during Co-production of Responsible AI Values
- URL: http://arxiv.org/abs/2307.10221v1
- Date: Fri, 14 Jul 2023 21:57:46 GMT
- Title: `It is currently hodgepodge'': Examining AI/ML Practitioners' Challenges
during Co-production of Responsible AI Values
- Authors: Rama Adithya Varanasi, Nitesh Goyal
- Abstract summary: We interviewed 23 individuals, across 10 organizations, tasked to ship AI/ML based products while upholding RAI norms.
Top-down and bottom-up institutional structures create burden for different roles preventing them from upholding RAI values.
We recommend recommendations for inclusive and equitable RAI value-practices.
- Score: 4.091593765662773
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Recently, the AI/ML research community has indicated an urgent need to
establish Responsible AI (RAI) values and practices as part of the AI/ML
lifecycle. Several organizations and communities are responding to this call by
sharing RAI guidelines. However, there are gaps in awareness, deliberation, and
execution of such practices for multi-disciplinary ML practitioners. This work
contributes to the discussion by unpacking co-production challenges faced by
practitioners as they align their RAI values. We interviewed 23 individuals,
across 10 organizations, tasked to ship AI/ML based products while upholding
RAI norms and found that both top-down and bottom-up institutional structures
create burden for different roles preventing them from upholding RAI values, a
challenge that is further exacerbated when executing conflicted values. We
share multiple value levers used as strategies by the practitioners to resolve
their challenges. We end our paper with recommendations for inclusive and
equitable RAI value-practices, creating supportive organizational structures
and opportunities to further aid practitioners.
Related papers
- Using Case Studies to Teach Responsible AI to Industry Practitioners [8.152080071643685]
We propose a novel stakeholder-first educational approach that uses interactive case studies to achieve organizational and practitioner -level engagement and advance learning of Responsible AI (RAI)
Our assessment results indicate that participants found the workshops engaging and reported a positive shift in understanding and motivation to apply RAI to their work.
arXiv Detail & Related papers (2024-07-19T22:06:06Z) - The Impossibility of Fair LLMs [59.424918263776284]
The need for fair AI is increasingly clear in the era of large language models (LLMs)
We review the technical frameworks that machine learning researchers have used to evaluate fairness.
We develop guidelines for the more realistic goal of achieving fairness in particular use cases.
arXiv Detail & Related papers (2024-05-28T04:36:15Z) - A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law [65.87885628115946]
Large language models (LLMs) are revolutionizing the landscapes of finance, healthcare, and law.
We highlight the instrumental role of LLMs in enhancing diagnostic and treatment methodologies in healthcare, innovating financial analytics, and refining legal interpretation and compliance strategies.
We critically examine the ethics for LLM applications in these fields, pointing out the existing ethical concerns and the need for transparent, fair, and robust AI systems.
arXiv Detail & Related papers (2024-05-02T22:43:02Z) - CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledge [69.82940934994333]
We introduce CulturalTeaming, an interactive red-teaming system that leverages human-AI collaboration to build challenging evaluation dataset.
Our study reveals that CulturalTeaming's various modes of AI assistance support annotators in creating cultural questions.
CULTURALBENCH-V0.1 is a compact yet high-quality evaluation dataset with users' red-teaming attempts.
arXiv Detail & Related papers (2024-04-10T00:25:09Z) - Large Multimodal Agents: A Survey [78.81459893884737]
Large language models (LLMs) have achieved superior performance in powering text-based AI agents.
There is an emerging research trend focused on extending these LLM-powered AI agents into the multimodal domain.
This review aims to provide valuable insights and guidelines for future research in this rapidly evolving field.
arXiv Detail & Related papers (2024-02-23T06:04:23Z) - Towards Equitable Agile Research and Development of AI and Robotics [0.0]
We propose a framework for adapting widely practiced Research and Development (R&D) project management methodologies to build organizational equity capabilities.
We describe how project teams can organize and operationalize the most promising practices, skill sets, organizational cultures, and methods to detect and address rights-based fairness, equity, accountability, and ethical problems.
arXiv Detail & Related papers (2024-02-13T06:13:17Z) - POLARIS: A framework to guide the development of Trustworthy AI systems [3.02243271391691]
There is a significant gap between high-level AI ethics principles and low-level concrete practices for AI professionals.
We develop a novel holistic framework for Trustworthy AI - designed to bridge the gap between theory and practice.
Our goal is to empower AI professionals to confidently navigate the ethical dimensions of Trustworthy AI.
arXiv Detail & Related papers (2024-02-08T01:05:16Z) - Investigating Responsible AI for Scientific Research: An Empirical Study [4.597781832707524]
The push for Responsible AI (RAI) in such institutions underscores the increasing emphasis on integrating ethical considerations within AI design and development.
This paper aims to assess the awareness and preparedness regarding the ethical risks inherent in AI design and development.
Our results have revealed certain knowledge gaps concerning ethical, responsible, and inclusive AI, with limitations in awareness of the available AI ethics frameworks.
arXiv Detail & Related papers (2023-12-15T06:40:27Z) - Responsible AI Considerations in Text Summarization Research: A Review
of Current Practices [89.85174013619883]
We focus on text summarization, a common NLP task largely overlooked by the responsible AI community.
We conduct a multi-round qualitative analysis of 333 summarization papers from the ACL Anthology published between 2020-2022.
We focus on how, which, and when responsible AI issues are covered, which relevant stakeholders are considered, and mismatches between stated and realized research goals.
arXiv Detail & Related papers (2023-11-18T15:35:36Z) - Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation [52.930183136111864]
We propose using scorable negotiation to evaluate Large Language Models (LLMs)
To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities.
We provide procedures to create new games and increase games' difficulty to have an evolving benchmark.
arXiv Detail & Related papers (2023-09-29T13:33:06Z) - Where Responsible AI meets Reality: Practitioner Perspectives on
Enablers for shifting Organizational Practices [3.119859292303396]
This paper examines and seeks to offer a framework for analyzing how organizational culture and structure impact the effectiveness of responsible AI initiatives in practice.
We present the results of semi-structured qualitative interviews with practitioners working in industry, investigating common challenges, ethical tensions, and effective enablers for responsible AI initiatives.
arXiv Detail & Related papers (2020-06-22T15:57:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.