Psycholinguistic Analyses in Software Engineering Text: A Systematic Literature Review

• URL: http://arxiv.org/abs/2503.05992v2
• Date: Thu, 17 Apr 2025 18:05:41 GMT
• Title: Psycholinguistic Analyses in Software Engineering Text: A Systematic Literature Review
• Authors: Amirali Sajadi, Kostadin Damevski, Preetha Chatterjee,
• Abstract summary: Linguistic Inquiry and Word Count (LIWC) offer clearer, interpretable insights into cognitive and emotional processes exhibited in text.<n>Despite its wide use in software engineering research, no comprehensive review of LIWC's use has been conducted.<n>We conducted a systematic review of six prominent databases, identifying 43 SE-related papers using LIWC.
• Score: 9.229310642804036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context: A deeper understanding of human factors in software engineering (SE) is essential for improving team collaboration, decision-making, and productivity. Communication channels like code reviews and chats provide insights into developers' psychological and emotional states. While large language models excel at text analysis, they often lack transparency and precision. Psycholinguistic tools like Linguistic Inquiry and Word Count (LIWC) offer clearer, interpretable insights into cognitive and emotional processes exhibited in text. Despite its wide use in SE research, no comprehensive review of LIWC's use has been conducted. Objective: We examine the importance of psycholinguistic tools, particularly LIWC, and provide a thorough analysis of its current and potential future applications in SE research. Methods: We conducted a systematic review of six prominent databases, identifying 43 SE-related papers using LIWC. Our analysis focuses on five research questions. Results: Our findings reveal a wide range of applications, including analyzing team communication to detect developer emotions and personality, developing ML models to predict deleted Stack Overflow posts, and more recently comparing AI-generated and human-written text. LIWC has been primarily used with data from project management platforms (e.g., GitHub) and Q&A forums (e.g., Stack Overflow). Key BSE concepts include Communication, Organizational Climate, and Positive Psychology. 26 of 43 papers did not formally evaluate LIWC. Concerns were raised about some limitations, including difficulty handling SE-specific vocabulary. Conclusion: We highlight the potential of psycholinguistic tools and their limitations, and present new use cases for advancing the research of human factors in SE (e.g., bias in human-LLM conversations).

Related papers

• How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses [2.681175517296992]
  We analyze essay responses from a free university-level MOOC on AI ethics. We find that the launch of ChatGPT coincided with significant changes in both the length and style of student essays. We also observe -- as expected based on related public discourse -- changes in prevalence of key content words related to AI and LLMs.
  arXiv  Detail & Related papers  (2025-04-17T15:51:59Z)
• Comparing Native and Non-native English Speakers' Behaviors in Collaborative Writing through Visual Analytics [22.95323370401823]
  textscCOALA is a novel visual analytics tool that improves model interpretability by displaying uncertainties in author clusters.<n>We present the insights discovered by participants using textscCOALA, suggest features for future AI-assisted collaborative writing tools, and discuss the broader implications for analyzing collaborative processes beyond writing.
  arXiv  Detail & Related papers  (2025-02-25T22:39:55Z)
• Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMs [50.0874045899661]
  We introduce CharacterBot, a model designed to replicate both the linguistic patterns and distinctive thought processes of a character.<n>Using Lu Xun as a case study, we propose four training tasks derived from his 17 essay collections.<n>These include a pre-training task focused on mastering external linguistic structures and knowledge, as well as three fine-tuning tasks.<n>We evaluate CharacterBot on three tasks for linguistic accuracy and opinion comprehension, demonstrating that it significantly outperforms the baselines on our adapted metrics.
  arXiv  Detail & Related papers  (2025-02-18T16:11:54Z)
• Sentiment Analysis Tools in Software Engineering: A Systematic Mapping Study [43.44042227196935]
  We aim to help developers or stakeholders in their choice of sentiment analysis tools for their specific purpose.<n>Our results summarize insights from 106 papers with respect to (1) the application domain, (2) the purpose, (3) the used data sets, (4) the approaches for developing sentiment analysis tools, (5) the usage of already existing tools, and (6) the difficulties researchers face.
  arXiv  Detail & Related papers  (2025-02-11T19:02:25Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [55.33653554387953]
  Pattern Analysis and Machine Intelligence (PAMI) has led to numerous literature reviews aimed at collecting and fragmented information.<n>This paper presents a thorough analysis of these literature reviews within the PAMI field.<n>We try to address three core research questions: (1) What are the prevalent structural and statistical characteristics of PAMI literature reviews; (2) What strategies can researchers employ to efficiently navigate the growing corpus of reviews; and (3) What are the advantages and limitations of AI-generated reviews compared to human-authored ones.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• SOUL: Towards Sentiment and Opinion Understanding of Language [96.74878032417054]
  We propose a new task called Sentiment and Opinion Understanding of Language (SOUL) SOUL aims to evaluate sentiment understanding through two subtasks: Review (RC) and Justification Generation (JG)
  arXiv  Detail & Related papers  (2023-10-27T06:48:48Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis [6.596002578395151]
  ChatGPT is a new product of OpenAI and has emerged as the most popular AI product. This study explores the use of ChatGPT as a tool for data labeling for different sentiment analysis tasks.
  arXiv  Detail & Related papers  (2023-06-18T12:20:42Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• You Are What You Talk About: Inducing Evaluative Topics for Personality Analysis [0.0]
  evaluative language data has become more accessible with social media's rapid growth. We introduce the notion of evaluative topics, obtained by applying topic models to pre-filtered evaluative text. We then link evaluative topics to individual text authors to build their evaluative profiles.
  arXiv  Detail & Related papers  (2023-02-01T15:04:04Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• A Review on Text-Based Emotion Detection -- Techniques, Applications, Datasets, and Future Directions [4.257210316104905]
  The paper presents a systematic literature review of the existing literature published between 2005 to 2021 in text-based emotion detection. This review has meticulously examined 63 research papers from IEEE, Science Direct, Scopus, and Web of Science databases to address four primary research questions. An overview of various emotion models, techniques, feature extraction methods, datasets, and research challenges with future directions has also been represented.
  arXiv  Detail & Related papers  (2022-04-26T15:20:00Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.