Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor

• URL: http://arxiv.org/abs/2506.14652v1
• Date: Tue, 17 Jun 2025 15:44:41 GMT
• Title: Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
• Authors: Alexandra Olteanu, Su Lin Blodgett, Agathe Balayn, Angelina Wang, Fernando Diaz, Flavio du Pin Calmon, Margaret Mitchell, Michael Ekstrand, Reuben Binns, Solon Barocas,
• Abstract summary: We argue that a broader conception of what rigorous AI research and practice should entail is needed.<n>We aim to provide useful language and a framework for much-needed dialogue about the AI community's work.
• Score: 83.99510317617694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical, or computational methods are correctly applied. We argue that this narrow conception of rigor has contributed to the concerns raised by the responsible AI community, including overblown claims about AI capabilities. Our position is that a broader conception of what rigorous AI research and practice should entail is needed. We believe such a conception -- in addition to a more expansive understanding of (1) methodological rigor -- should include aspects related to (2) what background knowledge informs what to work on (epistemic rigor); (3) how disciplinary, community, or personal norms, standards, or beliefs influence the work (normative rigor); (4) how clearly articulated the theoretical constructs under use are (conceptual rigor); (5) what is reported and how (reporting rigor); and (6) how well-supported the inferences from existing evidence are (interpretative rigor). In doing so, we also aim to provide useful language and a framework for much-needed dialogue about the AI community's work by researchers, policymakers, journalists, and other stakeholders.

Related papers

• Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence [59.07578850674114]
  Sound deductive reasoning is an indisputably desirable aspect of general intelligence.<n>It is well-documented that even the most advanced frontier systems regularly and consistently falter on easily-solvable reasoning tasks.<n>We argue that their unsound behavior is a consequence of the statistical learning approach powering their development.
  arXiv  Detail & Related papers  (2025-06-30T14:37:50Z)
• Position: We Need Responsible, Application-Driven (RAD) AI Research [5.472297350890938]
  This position paper argues that achieving meaningful scientific and societal advances with artificial intelligence (AI) requires a responsible, application-driven approach (RAD) to AI research.<n>We present a vision for the future of application-driven AI research to unlock new value through technically feasible methods.
  arXiv  Detail & Related papers  (2025-05-07T03:43:52Z)
• Artificial Expert Intelligence through PAC-reasoning [21.91294369791479]
  Artificial Expert Intelligence (AEI) seeks to transcend the limitations of both Artificial General Intelligence (AGI) and narrow AI.<n>AEI seeks to integrate domain-specific expertise with critical, precise reasoning capabilities akin to those of top human experts.
  arXiv  Detail & Related papers  (2024-12-03T13:25:18Z)
• Establishing and Evaluating Trustworthy AI: Overview and Research Challenges [4.806063079434686]
  Some AI systems have yielded unexpected or undesirable outcomes or have been used in questionable manners. This paper synthesizes existing conceptualizations of trustworthy AI along six requirements. It aims to serve as a reference for a broad audience and as a basis for future research directions.
  arXiv  Detail & Related papers  (2024-11-15T06:05:52Z)
• Using AI Alignment Theory to understand the potential pitfalls of regulatory frameworks [55.2480439325792]
  This paper critically examines the European Union's Artificial Intelligence Act (EU AI Act) Uses insights from Alignment Theory (AT) research, which focuses on the potential pitfalls of technical alignment in Artificial Intelligence. As we apply these concepts to the EU AI Act, we uncover potential vulnerabilities and areas for improvement in the regulation.
  arXiv  Detail & Related papers  (2024-10-10T17:38:38Z)
• Beyond principlism: Practical strategies for ethical AI use in research practices [0.0]
  The rapid adoption of generative artificial intelligence in scientific research has outpaced the development of ethical guidelines. Existing approaches offer little practical guidance for addressing ethical challenges of AI in scientific research practices. I propose a user-centered, realism-inspired approach to bridge the gap between abstract principles and day-to-day research practices.
  arXiv  Detail & Related papers  (2024-01-27T03:53:25Z)
• Report of the 1st Workshop on Generative AI and Law [78.62063815165968]
  This report presents the takeaways of the inaugural Workshop on Generative AI and Law (GenLaw) A cross-disciplinary group of practitioners and scholars from computer science and law convened to discuss the technical, doctrinal, and policy challenges presented by law for Generative AI.
  arXiv  Detail & Related papers  (2023-11-11T04:13:37Z)
• Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness [55.037030060643126]
  Intersectionality is a critical framework that allows us to examine how social inequalities persist. We argue that adopting intersectionality as an analytical framework is pivotal to effectively operationalizing fairness.
  arXiv  Detail & Related papers  (2023-03-16T21:02:09Z)
• Artificial intelligence in government: Concepts, standards, and a unified framework [0.0]
  Recent advances in artificial intelligence (AI) hold the promise of transforming government. It is critical that new AI systems behave in alignment with the normative expectations of society.
  arXiv  Detail & Related papers  (2022-10-31T10:57:20Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.