Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation

• URL: http://arxiv.org/abs/2602.18483v1
• Date: Tue, 10 Feb 2026 20:25:35 GMT
• Title: Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation
• Abstract summary: We examine the data practices and standards that underpin this work.<n>Because adversarial datasets determine the scope and accuracy of model evaluations, they are critical artifacts for assessing potential harms from large language models.
• Score: 7.4922225493924826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, red teaming, with roots in security, has become a key evaluative approach to ensure the safety and reliability of Generative Artificial Intelligence. However, most existing work emphasizes technical benchmarks and attack success rates, leaving the socio-technical practices of how red teaming datasets are defined, created, and evaluated under-examined. Drawing on 22 interviews with practitioners who design and evaluate red teaming datasets, we examine the data practices and standards that underpin this work. Because adversarial datasets determine the scope and accuracy of model evaluations, they are critical artifacts for assessing potential harms from large language models. Our contributions are first, empirical evidence of practitioners conceptualizing red teaming and developing and evaluating red teaming datasets. Second, we reflect on how practitioners' conceptualization of risk leads to overlooking the context, interaction type, and user specificity. We conclude with three opportunities for HCI researchers to expand the conceptualization and data practices for red-teaming.

Related papers

• ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts [54.60525564599342]
  ConceptScope is a scalable and automated framework for analyzing visual datasets.<n>It categorizes concepts into target, context, and bias types based on their semantic relevance and statistical correlation to class labels.<n>It reliably detects known biases and uncovers previously unannotated ones.
  arXiv  Detail & Related papers  (2025-10-30T06:46:17Z)
• Rethinking Evaluation of Infrared Small Target Detection [105.59753496831739]
  This paper introduces a hybrid-level metric incorporating pixel- and target-level performance, proposing a systematic error analysis method, and emphasizing the importance of cross-dataset evaluation.<n>An open-source toolkit has be released to facilitate standardized benchmarking.
  arXiv  Detail & Related papers  (2025-09-21T02:45:07Z)
• An LLM-based multi-agent framework for agile effort estimation [11.458115351010699]
  Effort estimation is a crucial activity in agile software development, where teams collaboratively review, discuss, and estimate the effort required to complete user stories in a product backlog.<n>Current practices in agile effort estimation heavily rely on subjective assessments, leading to inaccuracies and inconsistencies in the estimates.<n>We propose a novel multi-agent framework for agile estimation that not only can produce estimates, but also can coordinate, communicate and discuss with human developers and other agents to reach a consensus.
  arXiv  Detail & Related papers  (2025-09-17T23:26:43Z)
• Societal Impacts Research Requires Benchmarks for Creative Composition Tasks [10.67427286900562]
  This position paper argues that benchmarks focused on creative composition tasks is a necessary step towards understanding the societal harms of AI-generated content.<n>We identify creative composition tasks as a prevalent usage category where users seek help with personal tasks that require everyday creativity.<n>We call for greater transparency in usage patterns to inform the development of new benchmarks that can effectively measure both the progress and the impacts of models with creative capabilities.
  arXiv  Detail & Related papers  (2025-04-09T03:12:16Z)
• Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective [111.58315434849047]
  robustness of neural information retrieval models (IR) models has garnered significant attention. We view the robustness of IR to be a multifaceted concept, emphasizing its necessity against adversarial attacks, out-of-distribution (OOD) scenarios and performance variance. We provide an in-depth discussion of existing methods, datasets, and evaluation metrics, shedding light on challenges and future directions in the era of large language models.
  arXiv  Detail & Related papers  (2024-07-09T16:07:01Z)
• Benchmark Early and Red Team Often: A Framework for Assessing and Managing Dual-Use Hazards of AI Foundation Models [0.2383122657918106]
  A concern about cutting-edge or "frontier" AI foundation models is that an adversary may use the models for preparing chemical, biological, radiological, nuclear, cyber, or other attacks. At least two methods can identify foundation models with potential dual-use capability; each has advantages and disadvantages. We propose a research and risk-management approach using a combination of methods including both open benchmarks and closed red team evaluations.
  arXiv  Detail & Related papers  (2024-05-15T20:28:15Z)
• Towards Red Teaming in Multimodal and Multilingual Translation [7.440772334845366]
  This paper presents the first study on human-based red teaming for Machine Translation. It marks a significant step towards understanding and improving the performance of translation models. We report lessons learned and provide recommendations for both translation models and red teaming drills.
  arXiv  Detail & Related papers  (2024-01-29T15:49:40Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• FLIRT: Feedback Loop In-context Red Teaming [79.63896510559357]
  We propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation.
  arXiv  Detail & Related papers  (2023-08-08T14:03:08Z)
• Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics [80.07271410743806]
  We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
  arXiv  Detail & Related papers  (2023-03-25T09:34:05Z)
• A Sentiment Analysis Dataset for Trustworthiness Evaluation [22.734197353027632]
  Deep learning models are often criticized to be untrustworthy due to the black-box problem. We release a novel and well-annotated sentiment analysis dataset to evaluate robustness and interpretability.
  arXiv  Detail & Related papers  (2021-08-30T11:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.