Large language models for crowd decision making based on prompt design strategies using ChatGPT: models, analysis and challenges

• URL: http://arxiv.org/abs/2403.15587v1
• Date: Fri, 22 Mar 2024 19:21:44 GMT
• Title: Large language models for crowd decision making based on prompt design strategies using ChatGPT: models, analysis and challenges
• Authors: Cristina Zuheros, David Herrera-Poyatos, Rosana Montes, Francisco Herrera,
• Abstract summary: Crowd Decision Making (CDM) is a methodology able to infer opinions and decisions from plain texts. Large Language Models (LLMs) lead us to explore new scenarios of automatically understand written texts. This paper analyzes the use of ChatGPT based on prompt design strategies to assist in CDM processes to extract opinions and make decisions.
• Score: 8.107295925954475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social Media and Internet have the potential to be exploited as a source of opinion to enrich Decision Making solutions. Crowd Decision Making (CDM) is a methodology able to infer opinions and decisions from plain texts, such as reviews published in social media platforms, by means of Sentiment Analysis. Currently, the emergence and potential of Large Language Models (LLMs) lead us to explore new scenarios of automatically understand written texts, also known as natural language processing. This paper analyzes the use of ChatGPT based on prompt design strategies to assist in CDM processes to extract opinions and make decisions. We integrate ChatGPT in CDM processes as a flexible tool that infer the opinions expressed in texts, providing numerical or linguistic evaluations where the decision making models are based on the prompt design strategies. We include a multi-criteria decision making scenario with a category ontology for criteria. We also consider ChatGPT as an end-to-end CDM model able to provide a general opinion and score on the alternatives. We conduct empirical experiments on real data extracted from TripAdvisor, the TripR-2020Large dataset. The analysis of results show a promising branch for developing quality decision making models using ChatGPT. Finally, we discuss the challenges of consistency, sensitivity and explainability associated to the use of LLMs in CDM processes, raising open questions for future studies.

Related papers

• Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey [66.166184609616]
  ChatGPT has opened up immense potential for applying large language models (LLMs) to text-centric multimodal tasks. It is still unclear how existing LLMs can adapt better to text-centric multimodal sentiment analysis tasks.
  arXiv  Detail & Related papers  (2024-06-12T10:36:27Z)
• QuaLLM: An LLM-based Framework to Extract Quantitative Insights from Online Forums [10.684484559041284]
  This study introduces QuaLLM, a novel framework to analyze and extract quantitative insights from text data on online forums. We applied this framework to analyze over one million comments from two Reddit's rideshare worker communities.
  arXiv  Detail & Related papers  (2024-05-08T18:20:03Z)
• Enhancing Decision Analysis with a Large Language Model: pyDecision a Comprehensive Library of MCDA Methods in Python [0.0]
  pyDecision library, implemented in Python, provides a comprehensive collection of multicriteria decision analysis methods. pyDecision has integrated ChatGPT, an advanced Large Language Model, where decision-makers can use ChatGPT to discuss and compare the outcomes of different methods.
  arXiv  Detail & Related papers  (2024-04-09T15:06:25Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• ChatGPT-HealthPrompt. Harnessing the Power of XAI in Prompt-Based Healthcare Decision Support using ChatGPT [15.973406739758856]
  This study presents an innovative approach to the application of large language models (LLMs) in clinical decision-making, focusing on OpenAI's ChatGPT. Our approach introduces the use of contextual prompts-strategically designed to include task description, feature description, and crucially, integration of domain knowledge-for high-quality binary classification tasks even in data-scarce scenarios.
  arXiv  Detail & Related papers  (2023-08-17T20:50:46Z)
• Building Emotional Support Chatbots in the Era of LLMs [64.06811786616471]
  We introduce an innovative methodology that synthesizes human insights with the computational prowess of Large Language Models (LLMs) By utilizing the in-context learning potential of ChatGPT, we generate an ExTensible Emotional Support dialogue dataset, named ExTES. Following this, we deploy advanced tuning techniques on the LLaMA model, examining the impact of diverse training strategies, ultimately yielding an LLM meticulously optimized for emotional support interactions.
  arXiv  Detail & Related papers  (2023-08-17T10:49:18Z)
• 'What are you referring to?' Evaluating the Ability of Multi-Modal Dialogue Models to Process Clarificational Exchanges [65.03196674816772]
  Referential ambiguities arise in dialogue when a referring expression does not uniquely identify the intended referent for the addressee. Addressees usually detect such ambiguities immediately and work with the speaker to repair it using meta-communicative, Clarification Exchanges (CE): a Clarification Request (CR) and a response. Here, we argue that the ability to generate and respond to CRs imposes specific constraints on the architecture and objective functions of multi-modal, visually grounded dialogue models.
  arXiv  Detail & Related papers  (2023-07-28T13:44:33Z)
• Improving Factuality and Reasoning in Language Models through Multiagent Debate [95.10641301155232]
  We present a complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer. Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a number of tasks. Our approach may be directly applied to existing black-box models and uses identical procedure and prompts for all tasks we investigate.
  arXiv  Detail & Related papers  (2023-05-23T17:55:11Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• Polling Latent Opinions: A Method for Computational Sociolinguistics Using Transformer Language Models [4.874780144224057]
  We use the capacity for memorization and extrapolation of Transformer Language Models to learn the linguistic behaviors of a subgroup within larger corpora of Yelp reviews. We show that even in cases where a specific keyphrase is limited or not present at all in the training corpora, the GPT is able to accurately generate large volumes of text that have the correct sentiment.
  arXiv  Detail & Related papers  (2022-04-15T14:33:58Z)
• Sentiment Analysis based Multi-person Multi-criteria Decision Making Methodology using Natural Language Processing and Deep Learning for Smarter Decision Aid. Case study of restaurant choice using TripAdvisor reviews [18.491161392558265]
  We propose the Sentiment Analysis based Multi-person Multi-criteria Decision Making (SA-MpMcDM) methodology for smarter decision aid. The SA-MpMcDM methodology incorporates an end-to-end multi-task deep learning model for aspect based sentiment analysis. We analyze the SA-MpMcDM methodology in different scenarios using and not using natural language and numerical evaluations.
  arXiv  Detail & Related papers  (2020-07-31T18:45:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.