COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities

• URL: http://arxiv.org/abs/2406.12074v3
• Date: Tue, 22 Oct 2024 06:38:07 GMT
• Title: COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities
• Authors: Zihao He, Minh Duc Chu, Rebecca Dorn, Siyi Guo, Kristina Lerman,
• Abstract summary: We introduce Community-Cross-Instruct, an unsupervised framework for aligning large language models to online communities to elicit beliefs. We demonstrate the method's utility in accurately representing political and diet communities on Reddit.
• Score: 5.0261645603931475
• License:
• Abstract: Social scientists use surveys to probe the opinions and beliefs of populations, but these methods are slow, costly, and prone to biases. Recent advances in large language models (LLMs) enable the creating of computational representations or "digital twins" of populations that generate human-like responses mimicking the population's language, styles, and attitudes. We introduce Community-Cross-Instruct, an unsupervised framework for aligning LLMs to online communities to elicit their beliefs. Given a corpus of a community's online discussions, Community-Cross-Instruct automatically generates instruction-output pairs by an advanced LLM to (1) finetune a foundational LLM to faithfully represent that community, and (2) evaluate the alignment of the finetuned model to the community. We demonstrate the method's utility in accurately representing political and diet communities on Reddit. Unlike prior methods requiring human-authored instructions, Community-Cross-Instruct generates instructions in a fully unsupervised manner, enhancing scalability and generalization across domains. This work enables cost-effective and automated surveying of diverse online communities.

Related papers

• A Multi-LLM Debiasing Framework [85.17156744155915]
  Large Language Models (LLMs) are powerful tools with the potential to benefit society immensely, yet, they have demonstrated biases that perpetuate societal inequalities. Recent research has shown a growing interest in multi-LLM approaches, which have been demonstrated to be effective in improving the quality of reasoning. We propose a novel multi-LLM debiasing framework aimed at reducing bias in LLMs.
  arXiv  Detail & Related papers  (2024-09-20T20:24:50Z)
• Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities [5.392300313326522]
  Large language models (LLMs) have shown promise in representing individuals and communities. This paper presents a framework for aligning LLMs with online communities via instruction-tuning. We demonstrate the utility of our approach by applying it to online communities centered on dieting and body image.
  arXiv  Detail & Related papers  (2024-08-18T05:41:36Z)
• Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration [84.47037877922293]
  Large language models (LLMs) struggle to model diverse preferences across cultures, demographics, and communities. We propose Modular Pluralism, a modular framework based on multi-LLM collaboration for pluralistic alignment. We evaluate Modular Pluralism with six tasks and four datasets featuring questions/instructions with value-laden and perspective-informed responses.
  arXiv  Detail & Related papers  (2024-06-22T22:07:40Z)
• Collective Constitutional AI: Aligning a Language Model with Public Input [20.95333081841239]
  There is growing consensus that language model (LM) developers should not be the sole deciders of LM behavior. We present Collective Constitutional AI (CCAI): a multi-stage process for sourcing and integrating public input into LMs. We demonstrate the real-world practicality of this approach by creating what is, to our knowledge, the first LM fine-tuned with collectively sourced public input.
  arXiv  Detail & Related papers  (2024-06-12T02:20:46Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• Aligning Large Language Models with Human: A Survey [53.6014921995006]
  Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect information. This survey presents a comprehensive overview of these alignment technologies, including the following aspects.
  arXiv  Detail & Related papers  (2023-07-24T17:44:58Z)
• Can AI Moderate Online Communities? [0.0]
  We use open-access generative pre-trained transformer models (GPT) from OpenAI to train large language models (LLM) Our preliminary findings suggest, that when properly trained, LLMs can excel in identifying actor intentions, moderating toxic comments, and rewarding positive contributions. We contribute to the information system (IS) discourse with a rapid development framework on the application of generative AI in content online moderation and management of culture in decentralized, pseudonymous communities.
  arXiv  Detail & Related papers  (2023-06-08T11:45:44Z)
• Training Socially Aligned Language Models on Simulated Social Interactions [99.39979111807388]
  Social alignment in AI systems aims to ensure that these models behave according to established societal values. Current language models (LMs) are trained to rigidly replicate their training corpus in isolation. This work presents a novel training paradigm that permits LMs to learn from simulated social interactions.
  arXiv  Detail & Related papers  (2023-05-26T14:17:36Z)
• Enhance Ambiguous Community Structure via Multi-strategy Community Related Link Prediction Method with Evolutionary Process [7.239725647907488]
  We design a new community attribute based link prediction strategy HAP. This paper aims at providing a community enhancement measure through adding links to clarify ambiguous community structures. The experimental results on twelve real-world datasets with ground truth community indicate that the proposed link prediction method outperforms other baseline methods.
  arXiv  Detail & Related papers  (2022-04-28T06:24:16Z)
• This Must Be the Place: Predicting Engagement of Online Communities in a Large-scale Distributed Campaign [70.69387048368849]
  We study the behavior of communities with millions of active members. We develop a hybrid model, combining textual cues, community meta-data, and structural properties. We demonstrate the applicability of our model through Reddit's r/place a large-scale online experiment.
  arXiv  Detail & Related papers  (2022-01-14T08:23:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.