Can ChatGPT Code Communication Data Fairly?: Empirical Evidence from Multiple Collaborative Tasks

• URL: http://arxiv.org/abs/2510.20584v1
• Date: Thu, 23 Oct 2025 14:09:03 GMT
• Title: Can ChatGPT Code Communication Data Fairly?: Empirical Evidence from Multiple Collaborative Tasks
• Authors: Jiangang Hao, Wenju Cui, Patrick Kyllonen, Emily Kerzabi,
• Abstract summary: This paper investigates ChatGPT-based automated coding of communication data using a typical coding framework for collaborative problem solving.<n>Our results show that ChatGPT-based coding exhibits no significant bias across gender and racial groups, paving the road for its adoption in large-scale assessment of collaboration and communication.
• Score: 0.7772234702556445
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Assessing communication and collaboration at scale depends on a labor intensive task of coding communication data into categories according to different frameworks. Prior research has established that ChatGPT can be directly instructed with coding rubrics to code the communication data and achieves accuracy comparable to human raters. However, whether the coding from ChatGPT or similar AI technology exhibits bias against different demographic groups, such as gender and race, remains unclear. To fill this gap, this paper investigates ChatGPT-based automated coding of communication data using a typical coding framework for collaborative problem solving, examining differences across gender and racial groups. The analysis draws on data from three types of collaborative tasks: negotiation, problem solving, and decision making. Our results show that ChatGPT-based coding exhibits no significant bias across gender and racial groups, paving the road for its adoption in large-scale assessment of collaboration and communication.

Related papers

• Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation [48.63200319578052]
  We develop a taxonomy of types of ambiguity that arise in this task and propose metrics to quantify them.<n>Our work also explores how multi-turn dialogue can reduce ambiguity, therefore, improve code accuracy by better matching user goals.
  arXiv  Detail & Related papers  (2025-10-10T13:44:40Z)
• Developing Shared Vocabulary System For Collaborative Software Engineering [0.0]
  The study was structured into three phases: problem identification, method development, and empirical validation.<n>Grounded Theory principles were employed to design a structured methodology for collaborative vocabulary development.<n> Empirical validation through controlled experiments demonstrated that while initial adoption introduced overhead, the shared vocabulary system significantly improved information density, documentation clarity, and collaboration efficiency over time.
  arXiv  Detail & Related papers  (2025-07-18T22:58:16Z)
• Automated Coding of Communications in Collaborative Problem-solving Tasks Using ChatGPT [4.2702945607449605]
  Collaborative problem solving (CPS) is widely recognized as a critical 21st-century skill.<n>We show that ChatGPT can code communication data to a satisfactory level.<n>We also show that refining prompts based on feedback from miscoded cases can improve coding accuracy.
  arXiv  Detail & Related papers  (2024-11-15T14:57:39Z)
• Pragmatic Communication in Multi-Agent Collaborative Perception [80.14322755297788]
  Collaborative perception results in a trade-off between perception ability and communication costs. We propose PragComm, a multi-agent collaborative perception system with two key components. PragComm consistently outperforms previous methods with more than 32.7K times lower communication volume.
  arXiv  Detail & Related papers  (2024-01-23T11:58:08Z)
• ChatGPT in the context of precision agriculture data analytics [0.19036571490366497]
  We argue that integrating ChatGPT into the data processing pipeline of automated sensors in precision agriculture has the potential to bring several benefits. We show three ways of how ChatGPT can interact with the database of the remote server. We examine the potential and the validity of the response of ChatGPT in analyzing, and interpreting agricultural data.
  arXiv  Detail & Related papers  (2023-11-10T20:44:30Z)
• Assessing the Promise and Pitfalls of ChatGPT for Automated Code Generation [2.0400340435492272]
  This paper presents a comprehensive evaluation of the code generation capabilities of ChatGPT, a prominent large language model. A dataset of 131 code-generation prompts across 5 categories was curated to enable robust analysis. Code solutions were generated by both ChatGPT and humans for all prompts, resulting in 262 code samples.
  arXiv  Detail & Related papers  (2023-11-05T12:56:40Z)
• Unmasking the giant: A comprehensive evaluation of ChatGPT's proficiency in coding algorithms and data structures [0.6990493129893112]
  We evaluate ChatGPT's ability to generate correct solutions to the problems fed to it, its code quality, and nature of run-time errors thrown by its code. We look into patterns in the test cases passed in order to gain some insights into how wrong ChatGPT code is in these kinds of situations.
  arXiv  Detail & Related papers  (2023-07-10T08:20:34Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Personalized Decentralized Multi-Task Learning Over Dynamic Communication Graphs [59.96266198512243]
  We propose a decentralized and federated learning algorithm for tasks that are positively and negatively correlated. Our algorithm uses gradients to calculate the correlations among tasks automatically, and dynamically adjusts the communication graph to connect mutually beneficial tasks and isolate those that may negatively impact each other. We conduct experiments on a synthetic Gaussian dataset and a large-scale celebrity attributes (CelebA) dataset.
  arXiv  Detail & Related papers  (2022-12-21T18:58:24Z)
• Training Conversational Agents with Generative Conversational Networks [74.9941330874663]
  We use Generative Conversational Networks to automatically generate data and train social conversational agents. We evaluate our approach on TopicalChat with automatic metrics and human evaluators, showing that with 10% of seed data it performs close to the baseline that uses 100% of the data.
  arXiv  Detail & Related papers  (2021-10-15T21:46:39Z)
• Learning to Match Jobs with Resumes from Sparse Interaction Data using Multi-View Co-Teaching Network [83.64416937454801]
  Job-resume interaction data is sparse and noisy, which affects the performance of job-resume match algorithms. We propose a novel multi-view co-teaching network from sparse interaction data for job-resume matching. Our model is able to outperform state-of-the-art methods for job-resume matching.
  arXiv  Detail & Related papers  (2020-09-25T03:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.