Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning

• URL: http://arxiv.org/abs/2409.04574v1
• Date: Fri, 6 Sep 2024 19:25:18 GMT
• Title: Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning
• Authors: Xinyue Liu, Harshita Diddee, Daphne Ippolito,
• Abstract summary: One-size-fits-all large language models (LLMs) are increasingly being used to help people with their writing. This paper explores whether parameter-efficient finetuning (PEFT) with Low-Rank Adaptation can effectively guide the style of LLM generations.
• Score: 24.263699489328427
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One-size-fits-all large language models (LLMs) are increasingly being used to help people with their writing. However, the style these models are trained to write in may not suit all users or use cases. LLMs would be more useful as writing assistants if their idiolect could be customized to match each user. In this paper, we explore whether parameter-efficient finetuning (PEFT) with Low-Rank Adaptation can effectively guide the style of LLM generations. We use this method to customize LLaMA-2 to ten different authors and show that the generated text has lexical, syntactic, and surface alignment with the target author but struggles with content memorization. Our findings highlight the potential of PEFT to support efficient, user-level customization of LLMs.

Related papers

• MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time [50.41806216615488]
  Large Language Models (LLMs) acquire extensive knowledge and remarkable abilities from extensive text corpora. To make LLMs more usable, aligning them with human preferences is essential. We propose an effective method, textbf MetaAlign, which aims to help LLMs dynamically align with various explicit or implicit preferences specified at inference time.
  arXiv  Detail & Related papers  (2024-10-18T05:31:13Z)
• CUTE: Measuring LLMs' Understanding of Their Tokens [54.70665106141121]
  Large Language Models (LLMs) show remarkable performance on a wide variety of tasks. This raises the question: To what extent can LLMs learn orthographic information? We propose a new benchmark, which features a collection of tasks designed to test the orthographic knowledge of LLMs.
  arXiv  Detail & Related papers  (2024-09-23T18:27:03Z)
• Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits [39.00434175773803]
  We hired professional writers to edit paragraphs in several creative domains. We curated the LAMP corpus: 1,057 LLM-generated paragraphs edited by professional writers according to our taxonomy. Analysis of LAMP reveals that none of the LLMs used in our study outperform each other in terms of writing quality.
  arXiv  Detail & Related papers  (2024-09-22T16:13:00Z)
• LLMs + Persona-Plug = Personalized LLMs [41.60364110693824]
  Personalization plays a critical role in numerous language tasks and applications, since users with the same requirements may prefer diverse outputs based on their individual interests. This has led to the development of various personalized approaches aimed at adapting large language models (LLMs) to generate customized outputs aligned with user preferences. We propose a novel personalized LLM model, ours. It constructs a user-specific embedding for each individual by modeling all her historical contexts through a lightweight plug-in user embedder module.
  arXiv  Detail & Related papers  (2024-09-18T11:54:45Z)
• Orchestrating LLMs with Different Personalizations [28.344891363780576]
  This paper presents a novel approach to aligning large language models (LLMs) with individual human preferences. Given stated preferences along multiple dimensions, such as helpfulness, conciseness, or humor, the goal is to create an LLM without re-training that best adheres to this specification. Starting from specialized expert LLMs, each trained for one particular preference dimension, we propose a black-box method that merges their outputs on a per-token level.
  arXiv  Detail & Related papers  (2024-07-04T22:55:02Z)
• One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models [67.49462724595445]
  Retrieval-augmented generation (RAG) is a promising way to improve large language models (LLMs) We propose a novel method that involves learning scalable and pluggable virtual tokens for RAG.
  arXiv  Detail & Related papers  (2024-05-30T03:44:54Z)
• CLAMP: Contrastive LAnguage Model Prompt-tuning [89.96914454453791]
  We show that large language models can achieve good image classification performance when adapted this way. Our approach beats state-of-the-art mLLMs by 13% and slightly outperforms contrastive learning with a custom text model.
  arXiv  Detail & Related papers  (2023-12-04T05:13:59Z)
• Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs [80.48606583629123]
  PASTA is a method that allows large language models to read text with user-specified emphasis marks. It can substantially enhance an LLM's ability to follow user instructions or integrate new knowledge from user inputs.
  arXiv  Detail & Related papers  (2023-11-03T22:56:43Z)
• PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts [16.59511985633798]
  Pretrained language models (PLMs) are powerful tools for capturing context. PLMs are typically pretrained and fine-tuned for universal use across different writers. This study aims to improve the accuracy of text understanding tasks by personalizing the fine-tuning of PLMs for specific writers.
  arXiv  Detail & Related papers  (2023-09-14T14:03:48Z)
• LLM-Rec: Personalized Recommendation via Prompting Large Language Models [62.481065357472964]
  Large language models (LLMs) have showcased their ability to harness commonsense knowledge and reasoning. Recent advances in large language models (LLMs) have showcased their remarkable ability to harness commonsense knowledge and reasoning. This study introduces a novel approach, coined LLM-Rec, which incorporates four distinct prompting strategies of text enrichment for improving personalized text-based recommendations.
  arXiv  Detail & Related papers  (2023-07-24T18:47:38Z)
• Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding [24.355564722047244]
  We show that current large language models struggle to capture some language styles without fine-tuning. We investigate whether LLMs can be meta-trained based on representative lexicons to recognize new styles they have not been fine-tuned on.
  arXiv  Detail & Related papers  (2023-05-24T00:17:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.