Challenges in designing research infrastructure software in multi-stakeholder contexts

• URL: http://arxiv.org/abs/2506.01492v1
• Date: Mon, 02 Jun 2025 09:50:30 GMT
• Title: Challenges in designing research infrastructure software in multi-stakeholder contexts
• Abstract summary: This study investigates the challenges in designing research infrastructure software for automated software publication in multi-stakeholder environments.<n>It examines technical, organizational, and social requirements across research software engineers (RSEs) and infrastructure facility staff (IFs)
• Score: 1.1786249372283562
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study investigates the challenges in designing research infrastructure software for automated software publication in multi-stakeholder environments, focusing specifically on the HERMES system. Through two quantitative surveys of research software engineers (RSEs) and infrastructure facility staff (IFs), it examines technical, organizational, and social requirements across these stakeholder groups. The study reveals significant differences in how RSEs and IFs prioritize various system features. While RSEs highly value compatibility with existing infrastructure, IFs prioritize user-focused aspects like system usability and documentation. The research identifies two main challenges in designing research infrastructure software: (1) the existence of multiple stakeholder groups with differing requirements, and (2) the internal heterogeneity within each stakeholder group across dimensions such as technical experience. The study also highlights that only half of RSE respondents actively practice software publication, pointing to potential cultural or technical barriers. Additionally, the research reveals discrepancies in how stakeholders view organizational aspects, with IFs consistently rating factors like responsibility structures and quality assurance as more important than RSEs do. These findings contribute to a better understanding of the complexities involved in designing research infrastructure software and emphasize the need for systems that can accommodate diverse user groups while maintaining usability across different technical expertise levels.

Related papers

• Report on the Designing Accountable Software Systems Workshop [18.019209577424792]
  The workshop was conducted as a collaborative systematization of knowledge that culminated in several research directions.<n>The findings include the importance of clarifying definitions and responsibilities within accountable organizations.<n>Further research was identified as needed to study the ways to improve the translation of accountability structures into the software design process.
  arXiv  Detail & Related papers  (2026-06-01T19:22:02Z)
• Towards Knowledgeable Deep Research: Framework and Benchmark [105.87257445617922]
  This paper introduces Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge.<n>We propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that integrates texts, figures, and tables into coherent multimodal reports.<n> Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics.
  arXiv  Detail & Related papers  (2026-04-09T02:06:27Z)
• GenAI for Systems: Recurring Challenges and Design Principles from Software to Silicon [62.2138479061386]
  Generative AI is reshaping how computing systems are designed, optimized, and built, yet research remains fragmented across software, architecture, and chip design communities.<n>This paper takes a cross-stack perspective, examining how generative models are being applied from code generation and distributed runtimes through hardware design space exploration to RTL synthesis, physical layout, and verification.
  arXiv  Detail & Related papers  (2026-02-16T22:45:33Z)
• A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System [56.40989626804489]
  This survey provides the first holistic analysis of Large Language Models-powered software engineering.<n>We review over 150 recent papers and propose a taxonomy along two key dimensions: (1) Solutions, categorized into prompt-based, fine-tuning-based, and agent-based paradigms, and (2) Benchmarks, including tasks such as code generation, translation, and repair.
  arXiv  Detail & Related papers  (2025-10-10T06:56:50Z)
• Towards a Taxonomy of Sustainability Requirements for Software Design [3.8168092489216385]
  This research conducts a Systematic Literature Review to extract and organize sustainability requirements from the state-of-the-art.<n>For each identified category, we provide clear definitions, associated metrics, and measures.<n>This systematized reference assists both software developers and researchers in effectively formulating, managing, and reconciling trade-offs within sustainable software development.
  arXiv  Detail & Related papers  (2025-10-10T04:20:11Z)
• Stakeholder Perspectives on Digital Twin Implementation Challenges in Healthcare: Insights from a Provider Digital Twin Case Study [0.0]
  This research investigates DT implementation challenges in healthcare by capturing the perspectives of four distinct stakeholders.<n>We conducted semi-structured interviews guided by the updated Consolidated Framework for Implementation Research (CFIR 2.0)<n>We then mapped each stakeholder group's preferences and concerns, revealing a nuanced landscape of converging and diverging perspectives.
  arXiv  Detail & Related papers  (2025-07-31T07:57:48Z)
• Benchmarking Multimodal Understanding and Complex Reasoning for ESG Tasks [56.350173737493215]
  Environmental, Social, and Governance (ESG) reports are essential for evaluating sustainability practices, ensuring regulatory compliance, and promoting financial transparency.<n>MMESGBench is a first-of-its-kind benchmark dataset to evaluate multimodal understanding and complex reasoning across structurally diverse and multi-source ESG documents.<n>MMESGBench comprises 933 validated QA pairs derived from 45 ESG documents, spanning across seven distinct document types and three major ESG source categories.
  arXiv  Detail & Related papers  (2025-07-25T03:58:07Z)
• A Systematic Mapping Study on Software Architecture for AI-based Mobility Systems [0.5156484100374057]
  We aim to provide the missing overview of existing architectures, their contribution to safety, and their level of maturity in AI-based safety-critical systems.<n>From a set of 1,639 primary studies, we selected 38 relevant studies dealing with safety assurance through software architecture in AI-based safety-critical systems.<n>The selected studies were then examined using various criteria to answer the research questions and identify gaps in this area of research.
  arXiv  Detail & Related papers  (2025-06-02T12:29:54Z)
• Creativity in LLM-based Multi-Agent Systems: A Survey [56.25583236738877]
  Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts.<n>This is the first survey dedicated to creativity in MAS.<n>We focus on text and image generation tasks, and present: (1) a taxonomy of agent proactivity and persona design; (2) an overview of generation techniques, including divergent exploration, iterative refinement, and collaborative synthesis, as well as relevant datasets and evaluation metrics; and (3) a discussion of key challenges, such as inconsistent evaluation standards, insufficient bias mitigation, coordination conflicts, and the lack of unified benchmarks.
  arXiv  Detail & Related papers  (2025-05-27T12:36:14Z)
• A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems [93.8285345915925]
  Reasoning is a fundamental cognitive process that enables logical inference, problem-solving, and decision-making.<n>With the rapid advancement of large language models (LLMs), reasoning has emerged as a key capability that distinguishes advanced AI systems.<n>We categorize existing methods along two dimensions: (1) Regimes, which define the stage at which reasoning is achieved; and (2) Architectures, which determine the components involved in the reasoning process.
  arXiv  Detail & Related papers  (2025-04-12T01:27:49Z)
• A Systematic Mapping Study on Architectural Approaches to Software Performance Analysis [8.629569588488328]
  This paper presents a systematic mapping study of 109 papers that integrate software architecture and performance analysis. We focus on five research questions that provide guidance for researchers and practitioners to gain an in-depth understanding of this research area.
  arXiv  Detail & Related papers  (2024-10-22T19:12:03Z)
• Making Software Development More Diverse and Inclusive: Key Themes, Challenges, and Future Directions [50.545824691484796]
  We identify six themes around the theme challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI)<n>We identify benefits, harms, and future research directions for the four main themes.<n>We discuss the remaining two themes, Artificial Intelligence & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• An Actionable Framework for Understanding and Improving Talent Retention as a Competitive Advantage in IT Organizations [44.342141516382284]
  This work presents an actionable framework for Talent Retention (TR) used in IT organizations. Our framework encompasses a set of factors, contextual characteristics, barriers, strategies, and coping mechanisms. Our findings indicated that software engineers can be differentiated from other professional groups.
  arXiv  Detail & Related papers  (2024-02-02T17:08:14Z)
• Enhancing Architecture Frameworks by Including Modern Stakeholders and their Views/Viewpoints [48.87872564630711]
  The stakeholders with data science and Machine Learning related concerns, such as data scientists and data engineers, are yet to be included in existing architecture frameworks.<n>We surveyed 61 subject matter experts from over 25 organizations in 10 countries.
  arXiv  Detail & Related papers  (2023-08-09T21:54:34Z)
• Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support [18.148737010217953]
  We conduct interviews and workshops with AI practitioners to identify practitioners' processes, challenges, and needs for support. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups. We identify impacts on fairness work stemming from a lack of engagement with direct stakeholders, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.
  arXiv  Detail & Related papers  (2021-12-10T17:14:34Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.