An instrument to measure factors that constitute the socio-technical context of testing experience

• URL: http://arxiv.org/abs/2505.01171v1
• Date: Fri, 02 May 2025 10:26:47 GMT
• Title: An instrument to measure factors that constitute the socio-technical context of testing experience
• Authors: Mark Swillus, Carolin Brandt, Andy Zaidman,
• Abstract summary: We consider testing a cooperative and social practice that is shaped by the tools developers use, the tests they write, and their mindsets and human needs.<n>This work is one part of a project that explores the human- and socio-technical context of testing.
• Score: 2.3992861500765543
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We consider testing a cooperative and social practice that is shaped by the tools developers use, the tests they write, and their mindsets and human needs. This work is one part of a project that explores the human- and socio-technical context of testing through the lens of those interwoven elements: test suite and tools as technical infrastructure and collaborative factors and motivation as mindset. Drawing on empirical observations of previous work, this survey examines how these factors relate to each other. We want to understand which combination of factors can help developers strive and make the most of their ambitions to leverage the potential that software testing practices have. In this report, we construct a survey instrument to measure the factors that constitute the socio-technical context of testing experience. In addition, we state our hypotheses about how these factors impact testing experience and explain the considerations and process that led to the construction of the survey questions.

Related papers

• Creativity in LLM-based Multi-Agent Systems: A Survey [56.25583236738877]
  Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts.<n>This is the first survey dedicated to creativity in MAS.<n>We focus on text and image generation tasks, and present: (1) a taxonomy of agent proactivity and persona design; (2) an overview of generation techniques, including divergent exploration, iterative refinement, and collaborative synthesis, as well as relevant datasets and evaluation metrics; and (3) a discussion of key challenges, such as inconsistent evaluation standards, insufficient bias mitigation, coordination conflicts, and the lack of unified benchmarks.
  arXiv  Detail & Related papers  (2025-05-27T12:36:14Z)
• Who cares about testing?: Co-creations of Socio-technical Software Testing Experiences [10.262884976304282]
  This study investigates the lived experience of software developers to illuminate how their opinions about testing change.<n>We construct a theory by systematically analyzing data from 19 in-depth, semi-structured interviews with software developers.<n>We develop eleven categories of circumstances that act as conditions for the application and adaptation of testing practices.
  arXiv  Detail & Related papers  (2025-04-09T18:35:36Z)
• Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge [78.35388859345056]
  We argue that the ML community would benefit from learning from and drawing on the social sciences when developing measurement instruments for evaluating GenAI systems.<n>We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI.
  arXiv  Detail & Related papers  (2025-02-01T21:09:51Z)
• Testing Research Software: An In-Depth Survey of Practices, Methods, and Tools [3.831549883667425]
  Testing research software is challenging due to the software's complexity and to the unique culture of the research software community.<n>This study focuses on test case design, challenges with expected outputs, use of quality metrics, execution methods, tools, and desired tool features.
  arXiv  Detail & Related papers  (2025-01-29T16:27:13Z)
• Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems [88.35461485731162]
  We identify four types of challenges that prevent practitioners from effectively using publicly available instruments for measuring representational harms. Our goal is to advance the development of instruments for measuring representational harms that are well-suited to practitioner needs.
  arXiv  Detail & Related papers  (2024-11-23T22:13:38Z)
• Evaluating Generative AI Systems is a Social Science Measurement Challenge [78.35388859345056]
  We present a framework for measuring concepts related to the capabilities, impacts, opportunities, and risks of GenAI systems. The framework distinguishes between four levels: the background concept, the systematized concept, the measurement instrument(s), and the instance-level measurements themselves.
  arXiv  Detail & Related papers  (2024-11-17T02:35:30Z)
• STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond [68.47402386668846]
  We introduce Structured Reasoning In Critical Text Assessment (STRICTA) to model text assessment as an explicit, step-wise reasoning process.<n>STRICTA breaks down the assessment into a graph of interconnected reasoning steps drawing on causality theory.<n>We apply STRICTA to a dataset of over 4000 reasoning steps from roughly 40 biomedical experts on more than 20 papers.
  arXiv  Detail & Related papers  (2024-09-09T06:55:37Z)
• Elevating Software Quality in Agile Environments: The Role of Testing Professionals in Unit Testing [0.0]
  Testing is an essential quality activity in the software development process. This paper explores the participation of test engineers in unit testing within an industrial context.
  arXiv  Detail & Related papers  (2024-03-20T00:41:49Z)
• InterEvo-TR: Interactive Evolutionary Test Generation With Readability Assessment [1.6874375111244329]
  We propose incorporating interactive readability assessments made by a tester into EvoSuite. Our approach, InterEvo-TR, interacts with the tester at different moments during the search. Our results show that the strategy to select and present intermediate results is effective for the purpose of readability assessment.
  arXiv  Detail & Related papers  (2024-01-13T13:14:29Z)
• Continuous Experimentation and Human Factors An Exploratory Study [4.419836325434071]
  The success of tools and systems relies heavily on their ability to meet the needs and expectations of users. User-centered design approaches, with a focus on human factors, have gained increasing attention as they prioritize the human element in the development process.
  arXiv  Detail & Related papers  (2023-11-01T14:56:33Z)
• LLM-based Interaction for Content Generation: A Case Study on the Perception of Employees in an IT department [85.1523466539595]
  This paper presents a questionnaire survey to identify the intention to use generative tools by employees of an IT company. Our results indicate a rather average acceptability of generative tools, although the more useful the tool is perceived to be, the higher the intention seems to be. Our analyses suggest that the frequency of use of generative tools is likely to be a key factor in understanding how employees perceive these tools in the context of their work.
  arXiv  Detail & Related papers  (2023-04-18T15:35:43Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• Active Inference in Robotics and Artificial Agents: Survey and Challenges [51.29077770446286]
  We review the state-of-the-art theory and implementations of active inference for state-estimation, control, planning and learning. We showcase relevant experiments that illustrate its potential in terms of adaptation, generalization and robustness.
  arXiv  Detail & Related papers  (2021-12-03T12:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.